markov chain monte carlo ( mcmc ) has become a standard tool in bayesian analysis .
the greatest benefit of mcmc is its generality  it is guaranteed to be consistent with virtually no assumptions on the underlying model . however , the practical applicability of mcmc generally depends on the dimension of the unknown variables , the number of data , and the computational resources available .
furthermore , because mcmc is sequential in nature , it can be difficult to implement efficiently with modern parallel and distributed computing architectures ; see @xcite for general discussion about mcmc in challenging scenarios , and with parallel computing architectures .
in recent years , a number of generic approximation methods have been developed for complex bayesian inference , for instance variational bayes ( cf .
reviews * ? ? ?
* ; * ? ? ?
* ) , expectation propagation @xcite and laplace approximations @xcite .
these methods have been reported to provide accurate enough inference for practical purposes in many scenarios , with a fraction of computing time compared with ( asymptotically ) exact approaches such as mcmc .
one problem with approximate methods is that the related bias is typically hard or impossible to quantify , and in practice the only way to ensure the validity of the method is to compare the outcome with an exact method , such as mcmc .
we promote , in broad terms , a well - known principle : the combination of an approximate method for ` preliminary ' inference , and importance sampling type correction of the output .
our primary focus is on the approximation of a marginal distribution ( or marginal likelihood ) and the use of mcmc for approximate marginal inference .
the outcome of the mcmc is corrected with importance sampling type estimates .
we detail minimal requirements which lead to strongly consistent estimators and give general conditions under which central limit theorems hold .
our setting highlights explicitly the connection of importance sampling type correction and recently developed pseudo - marginal type mcmc @xcite , such as particle mcmc @xcite , grouped independence metropolis - hastings ( gimh ) @xcite , approximate bayesian computation mcmc @xcite , the algorithm for estimation of discretely observed diffusions suggested in @xcite , and using annealed importance sampling @xcite .
it also enables the use of debiased estimators as suggested in @xcite .
all these methods are often computationally expensive , but our approach is based on independent realisations of such importance sampling type estimates , which can be calculated efficiently in parallel .
we focus on bayesian state space models , where importance sampling and particle filters are used for correction .
we believe that the setup is useful much more generally , for instance in latent variable models or discretisations of continuous models , such as diffusions or inverse problems .
importance sampling correction of mcmc has been suggested early in the mcmc literature ; see at least @xcite .
it has been used , for instance , when estimating bayes factors using a single mcmc output @xcite .
bhattacharya @xcite has suggested a consistent estimator based on regeneration and tan , doss and hobert @xcite give such estimators in case of using multiple markov chains .    using unbiased estimators of importance weights in this context
has been suggested at least in @xcite , who consider a generalisation of the pseudo - marginal method , allowing for likelihood estimators that occasionally take negative values .
this allows , for instance , using the recently discovered ` debiasing tricks ' @xcite in order to construct an unbiased estimator .
similar tricks , which would lead into non - negative unbiased estimators , as required by the pseudo - marginal method , may be unavailable ( cf . * ? ? ?
quiroz , villani and kohn @xcite applied the is correction in the data sub - sampling context .    nested sampling has also appeared in many forms in the monte carlo literature .
the smc^2^ algorithm @xcite is based on an application of nested sequential monte carlo steps , which has similarities with our framework , and the is^2^ method @xcite focuses on the case where the preliminary inference is based on importance sampling .
we focus on the mcmc approximation of the marginal distribution , which we believe often to be easily implementable in practice .
the markov chain nature of the marginal monte carlo approximation comes also with some extra theoretical issues which we address .
another specific feature of our work is that we connect is type correction explicitly with recently discovered ` exact approximation mcmc ' techniques @xcite .
theoretical advances @xcite have already led to more efficient implementation of such methods , but has also revealed fundamental limitations .
for instance , the methods may suffer from slow ( non - geometric ) convergence in practically interesting scenarios @xcite .
recently proposed correlated version of the pseudo - marginal mcmc @xcite may help in more efficient implementation of pseudo - marginal type methods , but the question of their efficient parallelisability remains a challenge .
the blocked parallelisable particle gibbs @xcite has appealing limiting properties , but its implementation still requires synchronisation between every update cycle , which may be costly in some computing environments .
we recognise that our is approach comes with its own limitations ( see section [ sec : discussion ] ) , but we believe that it can often be very useful with parallel computing facilities .
after preliminaries and notation in section [ sec : notation ] , we discuss a general importance sampling ( is ) type correction of mcmc outputs and we state related consistency results in section [ sec : simple ] .
we detail the general case where unbiased estimators are used ( proposition [ prop : simple - is ] ) and then extend the setting in order to allow inference in an extended state space ( proposition [ prop : proper - consistency ] ) .
the latter result is based on a general concept ( assumption [ a : proper ] ) , which we call ` proper weighting scheme ' following the terminology of liu @xcite .
it accommodates naturally estimators from importance sampling and sequential monte carlo .
we believe that assumption [ a : proper ] , or the more general assumption [ a : super - general ] , could be satisfied relatively easily in a wide variety of scenarios .    in section [ sec : ssm ] ,
we discuss how our importance sampling correction method applies in general state space models ( ssm ) , and observe that sequential importance sampling and particle filters both lead to proper weighting schemes ( corollary [ cor : proper - ssm ] ) .
the unbiasedness property of particle filters observed in @xcite suggests a method of its own interest : a simple , general and parallelisable general particle ssm smoothing algorithm , with naturally available consistent confidence intervals ( proposition [ prop : particle - smooth - confidence ] ) .
we are unaware of earlier works which explicitly state this potentially useful result , but we are mindful that it may be widely known by particle filter experts .
section [ sec : asvar - clt ] discusses conditions under which our estimators admit a central limit theorem .
the expressions for the asymptotic variance may be useful , for instance , when optimising the computational resources .
section [ sec : block ] focuses on a so - called jump chain representation @xcite , where we propose block estimators that use the jump chain structure . in this context , it is possible to employ variance reduction techniques , such as stratification or rao - blackwellisation using the scheme of @xcite . section [ sec : delayed ] starts by making an explicit connection between the proper weighting schemes and the pseudo - marginal type mcmc , and then between the is type correction of mcmc and a two - stage delayed acceptance scheme @xcite .    in section [ sec : exp ] , we detail a more specific class of ssms with linear - gaussian state dynamics and explain a generic laplace approximation method that fits this class of models . we compare different algorithmic variations with a model having poisson observations and with a stochastic volatility model . we conclude in section [ sec : discussion ] with a discussion of the possible implications of our findings .
throughout the paper , we consider general state spaces while using standard integral notation . if the model at hand is defined on euclidean space using standard probability densities , the following paragraph can be skipped .
each space @xmath0 is assumed to be equipped with a @xmath1-finite dominating measure ` @xmath2 ' on a @xmath1-algebra denoted with a corresponding calligraphic letter , such as @xmath3 .
product spaces are equipped with the related product @xmath1-algebras and product dominating measures .
if @xmath0 is a subset of an euclidean space @xmath4 , @xmath2 is taken by default as the lebesgue measure and @xmath3 as the borel subsets of @xmath0 .
if @xmath5 is a probability density on @xmath0 , we denote the support of @xmath5 as @xmath6 , and the probability measure corresponding to @xmath5 with the same symbol @xmath7 .
if @xmath8 , we denote @xmath9 , whenever well - defined . for a probability density or measure @xmath5 on @xmath0 and @xmath10 ,
we denote by @xmath11 the set of measurable @xmath8 with @xmath12 , and by @xmath13 the corresponding set of zero - mean functions .
if @xmath14 is a markov transition probability , we denote the probability measure @xmath15 , and the function @xmath16 .
iterates of transition probabilities are defined recursively through @xmath17 for @xmath18 .
we follow the convention @xmath19 , and if @xmath20 , we denote by @xmath21 the integers within the interval @xmath22 $ ] .
we use this notation in indexing , so that @xmath23 , @xmath24 .
if @xmath25 , then @xmath26 or @xmath27 is void , so that for example @xmath28 means @xmath29 .
similarly , if @xmath30 is a vector , then @xmath31 and @xmath32 . we also use double - indexing , and write @xmath33 .
the markov chain @xmath34 on @xmath0 is harris recurrent with invariant probability @xmath5 , if for every @xmath35 , and every initial distribution , the ergodic averages are strongly consistent , @xmath36 we call such a chain _ harris ergodic ( with respect to @xmath5)_.    recall that virtually all mcmc schemes are harris ergodic ( cf .
* ) , although in some cases careless implementation could lead to non - harris chains ( cf .
hereafter , @xmath37 is a probability density on @xmath0 and represents an approximation of a probability density @xmath38 of interest .
[ a : mcmc - is ] the density @xmath37 and the related markov chain @xmath39 satisfy    a.   @xmath34 is harris ergodic with respect to @xmath37 .
b.   [ item : support ] @xmath40 . c.   @xmath41 , where @xmath42 is a constant .
assumption [ a : mcmc - is ] is equivalent to a seemingly more general statement , where @xmath38 and @xmath37 are probability measures with @xmath43 . in particular , taking @xmath37 as the dominating measure on @xmath0 , then one can define @xmath44 , and the density of @xmath38 is the corresponding radon - nikodym derivative @xmath45 .
if assumption [ a : mcmc - is ] holds and it is possible to calculate the unnormalised importance weight @xmath46 pointwise , the chain @xmath39 can be weighted in order to approximate @xmath47 for every @xmath48 , using ( self - normalised ) importance sampling ( e.g. * ? ? ?
* ; * ? ? ?
* ) @xmath49{n\to\infty } \frac{\pi_a(w_u f)}{\pi_a(w_u ) }      = \pi(f ) ,      \label{eq : mcmc - is}\ ] ] as the harris ergodicity guarantees the almost sure convergence of both the numerator and the denominator .    in many cases of interest ,
the importance weights @xmath46 can not be calculated or they are extremely costly to evaluate . instead , it is often possible to construct unbiased estimators of @xmath50 , which may be used in place of @xmath50 under mild conditions . in order to formalise such a setting ,
we formulate the following abstract setting , which accommodates further generalisations of importance sampling type corrections .
[ a : aug ] suppose @xmath34 is a harris ergodic markov chain on @xmath0 with invariant probability @xmath37 , and @xmath51 are conditionally independent given @xmath34 taking values on a space @xmath52 , such that the distribution of @xmath53 depends only upon the value of @xmath54 : @xmath55    in what follows , we consider several such augmented chains .
note that the mapping @xmath56 in is never needed in practice , but only introduced for theoretic purposes .
technically , @xmath56 is required to be a regular conditional distribution of @xmath53 given @xmath54 , but in practice , the concerns about the regularity of @xmath56 are usually unnecessary .
[ thm : aug - consistency ] suppose assumption [ a : aug ] holds .
then , @xmath57 is a harris ergodic markov chain on @xmath58 with invariant probability @xmath59    theorem [ thm : aug - consistency ] follows from lemma [ lem : aug - prop ] in appendix [ app : aug ] .
let us now consider the simple importance sampling type correction where @xmath50 are replaced with unbiased estimates @xmath60 .
this idea has been suggested in more specific contexts earlier at least in @xcite .    [ a : unbiased ] suppose assumption [ a : mcmc - is ] holds , @xmath61 are independent random variables on @xmath62 conditional on @xmath34 , the distribution of @xmath60 depends only on the value of @xmath54 , and satisfies @xmath63 { \overset{\mathrm{a.s.}}{=}}w_u(x)$ ] for @xmath37-almost every @xmath64 .    [ prop : simple - is ] suppose assumption [ a : unbiased ] holds and denote @xmath65 the estimator @xmath66 is strongly consistent , @xmath67 almost surely , if either of the following hold :    a.   @xmath68 almost surely and @xmath48 .
b.   @xmath8 is such that @xmath69 , where @xmath70 $ ] .
the results follow from theorem [ thm : aug - consistency ] , once we check that the functions @xmath71 and @xmath72 .    in proposition [ prop : simple -
is ] :    a.   the case @xmath68 coincides with the case where pseudo - marginal algorithms can be implemented , and provide similarly consistent estimators for all @xmath48 ; see proposition [ prop : proper - pseudo ]
. b.   when @xmath60 are allowed to take also negative values , we believe that @xmath66 might fail to be consistent for all @xmath48 , and an extra condition such as stated might be indeed necessary . c.   if the relative error in @xmath60 is bounded : there exists @xmath73 such that @xmath74 for all @xmath64 , then @xmath66 is guaranteed to be strongly consistent for all @xmath48 .
we then focus on our main scenario , where both @xmath50 and @xmath75 may be estimated .
such a scenario occurs when @xmath37 is approximation of a marginal density , which we discuss in detail after we state the following abstract assumption .
[ a : super - general ] suppose assumption [ a : mcmc - is ] holds , let @xmath48 and let @xmath76 be @xmath77-valued random variables conditionally independent given @xmath34 , such that the distribution of @xmath53 depends only on the value of @xmath54 , and satisfies @xmath78 = f(x)w_u(x)\qquad\text{and}\qquad      { \mathbb{e}}[w_k\mid x_k = x ] = w_u(x),\ ] ] for @xmath37-almost every @xmath64 , and such that @xmath79\ ] ] satisfies @xmath80 .
[ prop : super - general ] if assumption [ a : super - general ] holds , then @xmath81{n\to\infty } \pi(f).\ ] ]    follows from theorem [ thm : aug - consistency ] , because both @xmath82 and @xmath83 are in @xmath84 .    in the specific cases we consider later , we define @xmath85 , where @xmath86 are random variables which satisfy @xmath87 = c_w \pi(x ) f(x ) ] \qquad \text{and}\qquad { \mathbb{e}}[u_k\mid x_k = x ] = c_w \pi(x).\ ] ] in some cases , however , it may be that @xmath88 is unavailable , for instance if @xmath89 is a pseudo - marginal algorithm targeting @xmath37
. it may still be possible to estimate the ratio @xmath46 in an unbiased fashion .    in some scenarios
, it may be possible to use simply @xmath90 which are independent of @xmath60 ( or @xmath91 ) , with @xmath92=f(x)$ ]
. however , such @xmath90 may be more difficult to construct than dependent @xmath93 which satisfy assumption [ a : super - general ] .
it is clear that assumption [ a : unbiased ] and the assumptions in proposition [ prop : simple - is ] form a special case of assumption [ a : super - general ] , by taking @xmath94 .
the following scheme formulates a case where @xmath34 targets @xmath88 , which is an approximation of a marginal density @xmath95 , and the aim is inference over a joint target density @xmath96 on an extended state space @xmath97 .
the following assumption is similar to ( * ? ? ?
* definition 2.5.1 ) :    [ a : proper ] suppose assumption [ a : mcmc - is ] holds , @xmath98 and @xmath99 are conditionally independent given @xmath34 , and take values on @xmath100^m\times{\mathbb{r}}$ ] with @xmath101 for all @xmath102 , and such that for every @xmath103 , the random variables @xmath104 satisfy @xmath105 { \overset{\mathrm{a.s.}}{=}}w_u(x ) f^*(x)$ ] , where @xmath106 for @xmath37-almost every @xmath64 .
[ prop : proper - consistency ] suppose that assumption [ a : proper ] holds .
then , the estimator @xmath107 satisfies @xmath108 almost surely , if either of the following hold :    a.   [ item : proper - positive ] @xmath68 almost surely for all @xmath109 and @xmath103 . b.   [ item : proper - general ] @xmath103 and @xmath110\ ] ] satisfies @xmath80 .
this is a special case of proposition [ prop : super - general ] , because assumption [ a : proper ] together with , which is also implied by , implies assumption [ a : super - general ] .
the following immediate result records that importance sampling estimators fit assumption [ a : proper ] naturally .
[ prop : augmented - is ] suppose @xmath111 such that @xmath112 is a probability density for each @xmath64 and such that @xmath113 .
let @xmath114 and denote @xmath115 where @xmath42 is some constant , and let @xmath116 .
then , @xmath117 form a proper weighting scheme ( assumption [ a : proper ] ) , and therefore , for any @xmath103 , @xmath118{n\to\infty } \pi^*(f).\ ] ]    the next result shows that sub - sampling within a proper weighting scheme leads to a proper weighting scheme .
[ prop : randomise - proper ] suppose that @xmath117 forms a proper weighting scheme .
let @xmath119 be random variables conditionally independent of @xmath120 such that @xmath121 .
then , @xmath122 forms a proper weighting scheme .    proposition [ prop : randomise - proper ] follows by observing that @xmath123 is a proper weighting scheme , with @xmath124 .
proposition [ prop : randomise - proper ] implies that the estimator @xmath125{n\to\infty } \pi^*(f)\ ] ] under the conditions stated in proposition [ prop : proper - consistency ] .
such extra randomisation generally leads to a higher asymptotic variance , but may be preferable if the weighted sample @xmath126 needs to be stored for further analysis .    in case of non - negative weights ,
it is easy to see that the following convex combinations of proper weighting schemes remain proper .
[ prop : convex - proper ] suppose @xmath127 and @xmath128 forms a proper weighting scheme for each @xmath129 , @xmath130 are conditionally independent given @xmath34 in the sense of assumption [ a : aug ] , and @xmath131 almost surely for all @xmath132 .
then , the random variables @xmath133 form a proper weighting scheme @xmath134 for every @xmath135^n$ ] with @xmath136 .
we discuss the implication of proposition [ prop : augmented - is ] in case of state space models in section [ sec : ssm ] .
in such a setting , @xmath89 stands for the ( static ) parameters of the model , @xmath137 for the sampled latent state trajectories , and @xmath138 for the corresponding ( unnormalised ) importance weights .
we recall also the unbiasedness property of particle filters which leads to proper weighting schemes .
state space model ( ssm ) techniques are applied in a wide variety of applications ( cf . * ? ? ?
we assume the existence of a family of ssms indexed by a static parameters @xmath139 taking values on a space @xmath140 .
the model consists of a sequence of latent ( unobserved ) states @xmath141 taking values in @xmath142 , and observations @xmath143 taking values in @xmath144 .
more specifically , the latent states @xmath141 form a markov chain with initial density @xmath145 and state transition densities @xmath146 for @xmath147 .
the observations @xmath143 are conditionally independent given @xmath141 and follow the observation model @xmath148 .
this model defines the following probability density on @xmath149 : @xmath150 where , by convention , @xmath151 .
the inference is based on the so - called smoothing distribution @xmath152 where @xmath153 is a normalising constant .    in a bayesian state space model ,
the parameters @xmath139 are regarded random as well , with a prior density @xmath154 , which leads to the following joint density on @xmath155 : @xmath156 the ( full ) bayesian inference is based on the posterior distribution @xmath157 with overall normalising constant @xmath158 .
let us start with a description of a sequential importance sampling procedure , which requires user - supplied ` proposal ' distributions @xmath159 for @xmath160 .
[ alg : sis ] input : @xmath143 and @xmath161 .
a.   for @xmath160 , sample @xmath162 and calculate @xmath163    output : @xmath164 , where @xmath165 and @xmath166 .
if the following well - known support condition holds , we recall that algorithm [ alg : sis ] provides a weighting which is shown to lead to proper weighting in corollary [ cor : proper - ssm ] .
[ a : support ] suppose that @xmath167 for all @xmath160 , where @xmath168    [ prop : sis - proper ] suppose assumption [ a : support ] holds , then the random variables @xmath169 , @xmath170 and @xmath171 produced by algorithm [ alg : sis ] satisfy @xmath172      & = p^{(\theta)}(y_{1:t } ) \int p^{(\theta)}(\alpha_{1:t}\mid y_{1:t } )      f(\alpha_{1:t } ) { \mathrm{d}}\alpha_{1:t}.      \label{eq : ssm - proper}\end{aligned}\ ] ] for any function @xmath173 with the above integral well - defined .
proposition [ prop : sis - proper ] follows from proposition [ prop : augmented - is ] , because @xmath174 are ratios of @xmath175 and @xmath176 , which is the law of the independent trajectories @xmath177 .
let us consider next a particle filter algorithm @xcite ; see also the monographs @xcite , which is often much more efficient in practice .
the algorithm is based on same user - supplied proposal distributions @xmath178 as sis , but requires also a ` resampling ' distribution @xmath179 , which draws a sample @xmath180 from a discrete probability mass @xmath181 .
whenever the index ` @xmath182 ' appears in the algorithm , it takes values @xmath183 .
[ alg : pf ] input : @xmath143 and @xmath161 .
a.   sample @xmath184 , set @xmath185 and calculate @xmath186    for @xmath147 , do    a.   sample @xmath187 , where @xmath188 b.   sample @xmath189 and set @xmath190 . c.   calculate @xmath191    if @xmath192 in algorithm [ alg : pf ] , the algorithm may be terminated immediately  all the estimators considered in proposition [ prop : particle - proper ] below equal to zero then .
we prefer to avoid such explicit stopping mechanism for mathematical convenience .
we require the following well - known unbiasedness condition on the resampling procedure which is satisfied by the common multinomial , stratified , residual and systematic resampling methods ( cf . * ? ? ?
[ a : resampling ] the resampling procedure satisfies @xmath193=m\bar{\omega}^{(j)}.\ ] ] for any @xmath194 and any probability mass vector @xmath181 .
[ prop : particle - proper ] suppose that assumptions [ a : support ] and [ a : resampling ] hold , let @xmath161 and suppose @xmath173 is such that the integral in well - defined .
consider the random variables generated in algorithm [ alg : pf ] , and let @xmath195 .
a.   [ item : filter - smoother ] the random variables @xmath164 where @xmath196 and @xmath197 satisfy .
define for @xmath147 , and any @xmath198 , the backwards sampling probabilities @xmath199 and in case the sum is zero , set @xmath200 .
a.   [ item : backwards - sampling ] let @xmath201 be a random variable generated recursively by @xmath202 , and @xmath203 for @xmath204 .
for any @xmath103 , the random variables @xmath205 satisfy , where @xmath206 and @xmath207 . b.   [ item : fwd - bwd - smoothing ] if @xmath103 and @xmath208 for some @xmath209 , that is , @xmath210 is constant in all coordinates except @xmath211 and @xmath212 , then , the random variables @xmath213 satisfy ( with @xmath214 ) , where a.   @xmath215 , b.   @xmath216 , and where c.   [ item : smoothing - weights ] @xmath217 and @xmath218 for @xmath219 . c.   [ item : fwd - bwd - smoothing2 ] if @xmath103 and @xmath220 for some @xmath221 , then the random variables @xmath222 satisfy ( with @xmath214 ) , where @xmath223 and @xmath224 are defined in .    we provide a self - contained but concise proof of proposition [ prop : particle - proper ] in appendix
[ app : particle ] .
proposition [ prop : particle - proper ]    * corresponds to so - called filter - smoother @xcite .
this property was shown in ( * ? ? ?
* theorem 7.4.2 ) , although only in case of multinomial resampling .
the same property is behind the original version of the particle marginal metropolis - hastings @xcite ; see also remark [ rem : proper - mcmc ] .
+ the statement holds also when the particle filter is applied with a general sequence of distributions @xmath225 rather than the ssm @xcite .
* corresponds to backwards simulation , is analogous to the developments in conditional sequential monte carlo with backwards sampling @xcite .
this allows us to draw any number of independent trajectories ( indices ) , and average them , in order to satisfy . * and its special case correspond to the forward - backward smoother @xcite
; see also @xcite .
it can be seen as rao - blackwellised version of , but is applicable only when @xmath210 is a specific form .
this scheme can lead to lower variance , but it also has square complexity in @xmath226 , rendering it inapplicable with large @xmath226 .
we next note how propositions [ prop : sis - proper ] and [ prop : particle - proper ] ensure that both algorithms [ alg : sis ] and [ alg : pf ] yield a proper weighting scheme .
[ cor : proper - ssm ] let @xmath227 be a markov chain which is harris ergodic with respect to @xmath37 , and let @xmath228 correspond to    a.   independent runs of algorithm [ alg : sis ] with parameters @xmath227 , respectively , or b.   independent runs of a algorithm [ alg : pf ] with parameters @xmath227 , and the random variables defined in proposition [ prop : particle - proper ] or    then , the random variables @xmath229 with @xmath230 provide a proper weighting scheme for target distribution @xmath231 ( assumption [ a : proper ] ) .
similarly , if @xmath228 correspond to    a.   [ item : pf - marginal - inference ] independent runs of a algorithm [ alg : pf ] with parameters @xmath227 , and the random variables defined in proposition [ prop : particle - proper ] or ,    then @xmath229 provide a proper weighting scheme for the marginal distribution @xmath232 or @xmath233 , respectively .    corollary [ cor : proper - ssm ] is stated for a single marginal ( pair ) , but it is clear that we may estimate simultaneously several marginal ( pairs ) , so that proposition [ prop : particle - proper ] applies for every function which is of the form @xmath234 and proposition [ prop : particle - proper ] whenever @xmath235 .
it is possible to consider also more general classes of functions ; see for instance the discussion in @xcite .
we next discuss one implication of proposition [ prop : particle - proper ] outside the main focus of this paper .
namely , in the general state space smoothing context , that is , where @xmath161 is constant , proposition [ prop : particle - proper ] yields easily parallelisable particle smoothing algorithms , which also admit easily calculated consistent confidence interval estimates .
[ prop : particle - smooth - confidence ] suppose @xmath161 is fixed , and let @xmath228 correspond to independent random variables as defined in proposition [ prop : particle - proper ] .
a.   if the corresponding conditions of proposition [ prop : particle - proper ] are satisfied , then the estimator @xmath236{n\to\infty } \mu_f { \mathrel{\mathop:}=}\int      p^{(\theta)}(\alpha_{1:t}\mid y_{1:t } ) f(\alpha_{1:t}){\mathrm{d}}\alpha_{1:t},\ ] ] where @xmath237 , whenever the integral is well - defined . b.   if also @xmath238\big)^2\big]<\infty$ ] , then @xmath239 \xrightarrow[\mathrm{d}]{n\to\infty }      n(0,\sigma^2 ) ,      \qquad\text{where}\qquad      \sigma^2 { \mathrel{\mathop:}=}\frac{\sigma_*^2}{p^{(\theta)}(y_{1:t})^2}.\ ] ] c.   if in addition @xmath240<\infty$ ] , then @xmath241 , almost surely , where @xmath242\bigg)^2.\ ] ]    the result of proposition [ prop : particle - smooth - confidence ] is straightforward and probably widely known , but we highlight it here because of the practical consequences it has for parallel particle smoothing .
a.   proposition [ prop : particle - smooth - confidence ] ensures that , under the given conditions , the confidence intervals @xmath243 $ ] are asymptotically consistent , where @xmath244 corresponds to the desirable standard gaussian quantile .
b.   calculation of consistent confidence intervals for one realisation of a particle smoothing algorithm is not a straightforward task in general .
another recently suggested method @xcite relies on unbiased estimators obtained by coupling of conditional sequential monte carlo chains and randomisation as suggested by @xcite . c.
if the particle filter estimates satisfy a variance asymptotic such as @xmath245 where @xmath246 correspond to a particle filter run with @xmath226 particles , then if @xmath226 is taken sufficiently large , the estimate as in proposition [ prop : particle - smooth - confidence ] is nearly as efficient than the single particle filter estimate with @xmath247 particles , but benefits from easy parallel implementation .
d.   even estimates which have quadratic cost ( in @xmath226 ) , as in proposition [ prop : particle - proper ] and , may be of practical interest in some scenarios . then , choosing an optimal @xmath226 leads to a non - trivial optimisation task .
the asymptotic variance is a common efficiency criterion for markov chains .
when a central limit theorem ( clt ) holds , the limiting variance coincides with the asymptotic variance .
[ def : asvar ] suppose the markov chain @xmath34 on @xmath0 has transition probability @xmath14 which is harris ergodic with respect to invariant probability @xmath37
. for @xmath248 , the asymptotic variance of @xmath210 with respect to @xmath14 is @xmath249\bigg)^2,\ ] ] whenever the limit exists in @xmath250 $ ] , where @xmath251 stands for the _ stationary markov chain _ with transition probability @xmath14 , that is , with @xmath252 .
[ def : clt ] the markov chain @xmath34 satisfies a clt for @xmath210 , if @xmath253 , and for any initial distribution , @xmath254      \xrightarrow{n\to\infty } n\big(0,{\mathrm{var}}(f , p)\big),\ ] ] in distribution . in case @xmath255 , the limit is a unit mass at zero .    in order to guarantee a clt for a markov chain
, some form of rate of convergence is necessary .
we follow jones @xcite and define the following :    [ def : convergence - rate ] suppose that a markov transition probability @xmath14 is harris ergodic with respect to a probability measure @xmath5 , and @xmath256 and @xmath257 are such that @xmath258 if @xmath259 , we say that @xmath14 admits @xmath260 convergence rate .
different combinations of @xmath261 and @xmath262 imply a clt for different classes of functions ; see @xcite .
we record the following summary of conditions which guarantee a clt for the augmented markov chain .
the result follows easily from ( * ? ? ?
* theorem 9 ) , given the following simple result whose proof can be found in appendix [ app : clt ] .
[ lem : inherited - rate ] suppose that assumption [ a : aug ] holds , and the chain @xmath263 admits @xmath260 convergence rate .
then , the chain @xmath57 admits a @xmath264 convergence rate , where @xmath265 .
hereafter , we denote by @xmath266 the transition probability of @xmath267 and recall that @xmath268 is given in    [ prop : aug - clt ] suppose that assumption [ a : aug ] holds , and @xmath34 satisfies definition [ def : convergence - rate ] with some @xmath260 . if @xmath269 for some @xmath270 and if one of the following hold :    a.   [ item : uniform ] @xmath271 and @xmath34 is uniformly ergodic , that is , @xmath272 , b.   [ item : geometric ] @xmath273 for some @xmath274 and @xmath34 is geometrically ergodic , that is , definition [ def : convergence - rate ] holds with @xmath275 for some constants @xmath276 , c.   [ item : geometric - rev ] @xmath271 and @xmath34 is geometrically ergodic and reversible , d.   [ item : polynomial ] @xmath273 for some @xmath274 and @xmath34 is polynomially ergodic , @xmath277 with @xmath278 , and @xmath279 ,    then @xmath57 satisfies a clt for @xmath210 , with a finite limiting asymptotic variance @xmath280 where @xmath281 and @xmath282 are the conditional mean and variance , respectively : @xmath283    proposition [ prop : aug - clt ] is a consequence of lemma [ lem : inherited - rate ] , ( * ? ? ? * theorem 9 ) , and lemma [ lem : aug - prop ] and proposition [ prop : aug - asvar ] in appendix [ app : aug ] , because also reversibility of @xmath34 is inherited by @xmath57 .    if @xmath284 , then @xmath285 for all @xmath274 .
it follows that assuming @xmath286 in is enough to guarantee a clt for @xmath287 .
note that the @xmath288-geometric ergodicity can be shown to be inherited easily as well ( * ? ? ?
* lemma 45 ) .
let us finally apply these results to the estimator introduced in proposition [ prop : proper - consistency ] .    [
prop : proper - clt ] suppose conditions of proposition [ prop : proper - consistency ] are satisfied , which guarantee the strong consistency of @xmath66 given in , and suppose @xmath34 satisfies one of the conditions in proposition [ prop : aug - clt ] with @xmath289 . denote @xmath290.\ ] ] if @xmath291 , then @xmath292 \xrightarrow{n\to\infty } n(0,\sigma_f^2)\ ] ] in distribution , where @xmath293 and where @xmath294 and @xmath295 .
proof of proposition [ prop : proper - clt ] is given in appendix [ app : clt ] .
the latter term in the asymptotic variance expression of proposition [ prop : proper - clt ] contains the contribution of the importance sampling noise .
if the estimators @xmath296 are made increasingly accurate , in the sense that @xmath297 becomes negligible , the limiting case corresponds to an importance sampling corrected approximate mcmc , and evaluating the conditional expectation @xmath298 .
we conclude this section by noticing that the contribution of the importance sampling correction in asymptotic variance as in proposition [ prop : proper - clt ] can be estimated in a straightforward manner .
[ prop : importance - var ] suppose conditions of proposition [ prop : proper - consistency ] are satisfied , which guarantee the strong consistency of @xmath66 given in .
suppose also that @xmath299 where @xmath300 is defined in proposition [ prop : proper - clt ] , and @xmath301 , where @xmath302 $ ] .
then , the estimator @xmath303 satisfies @xmath304 almost surely as @xmath305 .
proof of proposition [ prop : importance - var ] is given in appendix [ app : clt ] .
the estimator @xmath306 in proposition [ prop : importance - var ] provides a _ lower bound _
estimate for the clt variance @xmath307 of @xmath66 .
it can provide useful information about the importance sampling noise contribution , and may be used as an optimisation criteria for the accuracy of the related estimators .
estimation of the full asymptotic variance @xmath308 , which includes the marginal chain contribution @xmath309 , is not a straightforward task in general ; see for example @xcite and references therein .
many mcmc algorithms include an accept - reject mechanism , which results in blocks of repeating values @xmath310 . when @xmath50 or @xmath311 are estimated , an obvious approach is to construct a single estimator which is used for the whole block .
this may allow for variance reduction , for instance when replacing simple independent importance sampling with block - wide stratified estimator , or using particle filter estimators as in proposition  [ prop : particle - proper ] , and choosing the number of particles proportional to the block length .
to formalise such an algorithm we consider the `` jump chain '' representation of @xmath34 ( cf .
[ def : jump ] suppose that @xmath34 is a markov chain with non - degenerate transition probability @xmath14 , that is , @xmath312 for all @xmath64 .
the jump chain @xmath313 with corresponding durations @xmath314 of @xmath34 is defined as follows : @xmath315 and @xmath316 , and then recursively @xmath317 where @xmath318 .
we note the following observations regarding definition [ def : jump ] :    a.   if @xmath14 is harris ergodic with respect to @xmath37 which is non - trivial ( that is @xmath319 for any @xmath64 ) , then necessarily @xmath312 for all @xmath64 , so definition [ def : jump ] is consistent . b.   if @xmath34 corresponds to a metropolis - hastings chain , with non - diagonal proposal distributions @xmath320 ( that is , @xmath321 for every @xmath64 ) , then the jump chain @xmath322 is simply the accepted states , and @xmath323 is the number of rejections occurred at state @xmath322 .
the importance sampling type estimator which we consider is of the following general form :    [ a : block - super - general ] suppose that assumption [ a : mcmc - is ] holds , let @xmath313 denote the corresponding jump chain ( definition [ def : jump ] ) , and assume @xmath48 .
suppose that @xmath324 are conditionally independent random variables on @xmath77 given @xmath313 , whose distribution depends only on the value of @xmath325 , and which satisfy @xmath326      = \frac{w_u(x)f(x)}{\alpha(x)}\qquad\text{and}\qquad      { \mathbb{e}}[\tilde{w}_k\mid \tilde{x}_k = x ] = \frac{w_u(x)}{\alpha(x)},\ ] ] for @xmath37-almost every @xmath64 , and @xmath327\ ] ] satisfies @xmath328 .
[ prop : block - super - general ] suppose assumption [ a : block - super - general ] holds , then @xmath329{n\to\infty } \pi(f).\ ] ]    proposition [ prop : jump - properties ] implies that the jump chain @xmath322 corresponding @xmath89 is harris ergodic with invariant probability @xmath330 .
the rest follows from proposition [ prop : super - general ] .
we consider next the most obvious block estimator , based on @xmath314 , corresponding to the jump chain and assumption [ a : super - general ] .
[ a : block - natural - general ] suppose that assumption [ a : mcmc - is ] holds , and let @xmath331 denote the corresponding jump chain ( definition [ def : jump ] ) .
let @xmath332 correspond to a proper weighting scheme ( assumption [ a : proper ] ) , such that the variables in the scheme may depend on both @xmath325 and @xmath333 , and such that @xmath334      = w_u(x)f^*(x),\ ] ] for all @xmath103 and @xmath37-almost every @xmath64 .
[ prop : block - natural - general ] suppose assumption [ a : block - natural - general ] holds , and @xmath335      \quad      \text{satisfies }      \quad      \pi_a(b)<\infty .
\label{eq : block - consistency - required}\ ] ] then , @xmath336{n\to\infty } \pi^*(f ) .
\label{eq : block - natural}\ ] ]    this follows from proposition [ prop : block - super - general ] , because the holding times @xmath337 are conditionally independent on @xmath322 and geometric with parameter @xmath338 , with expectation @xmath339 = 1/\alpha(x)$ ] .
therefore , @xmath340 and @xmath341 satisfy assumption [ a : block - super - general ] .    regarding assumption [ a : block - natural - general ] :    a.   the estimators
@xmath342 and @xmath60 which satisfy assumption [ a : block - natural - general ] may often come from a proper weighting scheme as in assumption [ a : proper ] , but here the parameter @xmath226 is allowed to depend on @xmath343 .
b.   the condition on @xmath344 $ ] is not optimal .
for instance , it is easy to find examples where the estimator is strongly consistent , even though the conditional expectation is unbounded in @xmath345 . in practice , however , the estimators are usually chosen either as independent of @xmath345 , or increasingly accurate in @xmath345 ; see remark [ rem : block - varying - or - not ] . c.   the estimator of proposition [ prop : augmented - is ] coincides with a block estimator where @xmath346 . however , the block estimator offers more flexibility . for instance
, @xmath347 could be stratified , leading to a smaller variance .
d.   even though we believe that the estimators of the form are often appropriate , we note that general form of the estimator as given in assumption [ a : block - super - general ] may be useful in some cases , in order to further reduce variance . for instance , assumption [ a : block - super - general ] accommodates the case where rao - blackwellised lower - variance estimators of @xmath348 are used instead of @xmath349 , as suggested in @xcite .
let us then consider the asymptotic variance of the estimator .
[ thm : block - clt ] suppose assumption [ a : block - natural - general ] holds , consider the estimator @xmath66 in and assume holds , @xmath350 , where @xmath351 , and @xmath352 , where @xmath353.\ ] ] if @xmath34 is @xmath288-geometrically ergodic , then @xmath354 \xrightarrow{n\to\infty } n(0,\sigma^2 )         \qquad\text{in distribution},\ ] ] where the limiting variance can be given as : @xmath355 ,      \label{eq : block - clt - var}\ ] ] where @xmath356,\ ] ] and @xmath14 stands for the markov transition probability of @xmath34 .
proof of theorem [ thm : block - clt ] is given in appendix [ app : jump ] .
[ rem : block - varying - or - not ] in theorem [ thm : block - clt ] asymptotic variance expression , the latter term corresponds to the contribution of noise in importance sampling estimates .
a.   when @xmath60 and @xmath357 correspond to @xmath333 independent estimates and @xmath358 , we might have @xmath359 , which leads to @xmath360 , so the contribution of the is would be upper bounded by @xmath361 .
b.   similarly , if @xmath362 , which could occur when @xmath93 are independent of @xmath333 , then , @xmath363 and the contribution of is would be upper bounded by @xmath364 .
the acceptance probability is lower bounded for geometrically ergodic chains @xcite , rendering the estimator a safe choice .    about the assumption of @xmath288-geometric ergodicity in theorem [ thm : block - clt ] :    a.   the asymptotic variance expression we derive for the ` numerator ' part of our estimator holds whenever @xmath34 is reversible , due to a result of deligiannidis and lee @xcite .
this does not , however , imply a clt .
b.   our proof relies on the existence of a solution @xmath365 to the poisson equation @xmath366 .
we believe that the jump chain @xmath313 inherits a central limit theorem from the base chain @xmath34 under general conditions , similarly as in section [ sec : asvar - clt ] .
however , the jump chain @xmath313 does not inherit a convergence rate from @xmath34 in general , so a simple approach such as in section [ sec : asvar - clt ] does not apply .
we start by making an explicit connection of proper weighting schemes with positive weights and pseudo - marginal type mcmc algorithms .
[ a : proper - pseudo ] suppose that @xmath367 is a probability density on @xmath97 , @xmath368 is a family of proposal densities on @xmath0 and @xmath369 is a family of probability distributions on @xmath370^m\times[0,\infty)$ ] such that for almost every @xmath64 and all @xmath103 , the variables @xmath371 satisfy @xmath372      = \varpi(x ) \int \pi^*(z\mid x )      f(x , z ) { \mathrm{d}}z,\ ] ] where @xmath373 .
[ alg : proper - pseudo ] suppose assumption [ a : proper - pseudo ] holds , let @xmath374 and @xmath375 and for @xmath102 iterate :    a.   draw @xmath376 and @xmath377 .
b.   with probability @xmath378 accept and set @xmath379 ; otherwise reject and set @xmath380 .
[ prop : proper - pseudo ] consider algorithm [ alg : proper - pseudo ] .
if @xmath381 is integrable and the markov chain @xmath57 is harris recurrent , then @xmath382{n\to\infty } \pi^*(f)\qquad\text{for        every $ f\in l^1(\pi^*)$,}\ ] ] if and only if assumption [ a : proper - pseudo ] holds with @xmath383 .    if @xmath381 is integrable
, then @xmath384 defines an unnormalised probability on @xmath385 .
define a composite proposal distribution @xmath386 , then it is straightforward to check that @xmath387 that is , algorithm [ alg : proper - pseudo ] is a metropolis - hastings with target @xmath388 , and therefore reversible with respect to @xmath389 .
if the chain is harris ergodic , then for every @xmath390 , the strong law of large numbers holds .
[ rem : proper - mcmc ] proposition [ prop : proper - pseudo ] connects proper weighting explicitly with the validity of pseudo - marginal type algorithms .
this analogue also suggests that our is type correction may be applied instead of any pseudo - marginal method .
the proper weighting formulation may also suggest variations of pseudo - marginal schemes .
for example , proposition [ prop : convex - proper ] suggests about the possibility of using several independent particle filters in a particle marginal metropolis - hastings update , which may be of interest in parallel computing .
let us then focus on delayed acceptance ( da ) mcmc @xcite , which is a multiple - stage acceptance method in metropolis - hastings algorithm , which is often based on an approximate target distributions in order to reduce computational effort of a metropolis - hastings algorithm .
we focus in particular on the two - stage da scheme based on an approximation @xmath37 of the true probability density @xmath38 , which has obvious similarities with the importance sampling scheme suggested earlier .
such da scheme with proposal densities @xmath368 can be summarised as follows :    [ alg : da ] suppose @xmath391 and for @xmath102 ,    a.   [ item : proposal ] generate a proposal @xmath392 b.   [ item : first - stage ] with probability @xmath393 continue to step , otherwise reject and set @xmath394 . here , @xmath395 c.   [ item : second - stage ] with probability @xmath396 set @xmath397 , otherwise reject and set @xmath398 .
the steps and alone would implement a regular metropolis - hastings step targeting @xmath37 , whereas the second - stage acceptance step alone would implement the accept - reject step of independence metropolis - hastings algorithm with proposal @xmath37 .
the rationale behind the da scheme is based on the same two assumptions which make the importance sampling corrected mcmc appealing : the evaluation of @xmath37 costs less than that of @xmath38 , and @xmath37 approximates @xmath38 reasonably well .
the former directly affects the expected cost of the algorithm : if @xmath399 is rejected at step , there is no need to calculate @xmath400 , implying lower expected cost .
the latter assumption guarantees that the second - stage acceptance rate is typically close to one .
even though the da version is guaranteed to be worse than direct metropolis - hastings targeting @xmath38 in the peskun sense @xcite , the lower cost often outweighs the difference by allowing a greater number of samples to be drawn with da .
we note that a proper weighting scheme ( assumption [ a : proper ] ) with non - negative weights leads to a valid da scheme .    [ a : da - proper ] suppose assumption [ a : proper - pseudo ] holds with @xmath401 and a @xmath37 is a probability density on @xmath0 with @xmath40 .
[ alg : da - proper ] suppose assumption [ a : da - proper ] holds , and @xmath402 are proposal densities as in algorithm [ alg : da ] .
define the markov chain @xmath267 with initial values @xmath403 and @xmath404 , and for @xmath102 as follows :    a.   generate a proposal @xmath392 .
b.   [ item : first - stage2 ] with probability @xmath393 continue to step , otherwise reject and set @xmath405 . c.   [ item : second - stage2 ]
generate @xmath406 , then with probability @xmath407 accept and set @xmath408 , otherwise reject and set @xmath380 .    [
prop : da - valid ] if assumption [ a : da - proper ] holds and @xmath402 are such that algorithm [ alg : da - proper ] defines a harris recurrent markov chain @xmath57 , then algorithm [ alg : da - proper ] satisfies @xmath409{n\to\infty } \pi^*(f ) ,      \qquad\text{for every $ f\in l^1(\pi^*)$.}\ ] ]    the proof of proposition [ prop : da - valid ] follows from @xcite , because algorithm [ alg : da - proper ] is a valid da version of algorithm [ alg : proper - pseudo ] .
proposition [ prop : da - valid ] highlights that when a da - mcmc scheme is of the pseudo - marginal form as algorithm [ alg : da - proper ] , the importance sampling type correction as we suggest is applicable , and a natural alternative for da - mcmc .
the efficiency of da - mcmc was studied by banterle , grazian , lee and robert @xcite .
sherlock , thiery and golithly @xcite studied the efficiency of pseudo - marginal with da as in algorithm [ alg : da - proper ] , with diffusion limit techniques .
contrary to the is scheme , it is much harder to study the effect of noisy importance weights with da in general , as the noise affects the markov chain convergence rate .
however , the authors @xcite are able to provide useful heuristic how to tune the accuracy of the pseudo - marginal da method .
we now turn into an illustration of our generic framework with more specific state space models .
section [ sec : lin - gauss - ssms ] discusses an important sub - class of state space models , where the state dynamics are linear - gaussian .
we describe a general method which is based on a gaussian approximation of the conditional smoothing distribution .
a special case of the general state space model introduced in section [ sec : ssm ] is a model where both @xmath142 and @xmath144 are euclidean , and the state dynamics @xmath410 are linear - gaussian , but the observational distribution @xmath411 may be non - gaussian and/or non - linear . more specifically , @xmath412 are defined through @xmath413 and @xmath414 our approach allows , for instance , exponential family observation models with gaussian , poisson , binomial , negative binomial , and gamma distributions , and a stochastic volatility model .
we discuss the specific models later in sections [ poisson ] and [ sv - model ] .
compared to the general state space form , this class of models may seem restrictive , but it still contains a large number of commonly used statistical models , such as ( generalised ) structural time series models , cubic splines , generalised linear models , and autoregressive integrated moving average ( arima ) models .
the benefit of working within this subset of general state space models is that we have access to approximations taking advantage of the specific structure of the model .
our approximation scheme , based on @xcite , relies on a laplace approximation @xmath415 of the conditional densities @xmath416 .
the approximating gaussian model is found by using an iterative process closely related to iterative reweighted least squares algorithm used in generalised linear models context @xcite . in the approximating gaussian model
, the observation equation is replaced by a linear - gaussian one @xmath417 where the pseudo - observations @xmath418 and their variances @xmath419 are based on the first and second derivatives of @xmath420 with respect to @xmath421 at @xmath422 , the conditional mode estimate of the approximate model @xcite .
our approximation of the marginal density @xmath37 is based on the marginal likelihood decomposition @xmath423,\ ] ] where @xmath424 , the terms @xmath425 correspond to the product of pseudo - observation densities , and the expectation is taken with respect to @xmath426 .
if the approximation @xmath427 is close to @xmath428 in the high - probability regions of @xmath426 , the expectation term is close to one , so we take @xmath429 our approximation defines , in fact , a joint approximate distribution @xmath430 , and we use @xmath431 as a proposal distribution in our importance sampling correction .
we use the laplace approximation @xmath432 described in section [ sec : lin - gauss - ssms ] as the importance distribution , and we use the simulation smoothing algorithm @xcite to simulate antithetic pairs of @xmath141 from @xmath432 .
we implemented the following inference algorithms in our experiments :    amcmc : :    approximate mcmc targeting @xmath433 , and for each    accepted @xmath139 , sampling one realisation from    @xmath434 with the    simulation smoother .
mcmc : :    exact pseudo - marginal mcmc ( algorithm [ alg : proper - pseudo ] ) algorithm    using importance sampling estimates with @xmath226 samples .
da : :    delayed acceptance version of the above ( algorithm [ alg : da - proper ] ) .
is1 : :    simple importance sampling estimator given in with @xmath226    samples .
is2 : :    blocked estimator as in , with is estimators based on @xmath226    samples , independent of @xmath435 . is - pf : :    this is similar to is2 but uses the bootstrap particle filter @xcite    ( algorithm [ alg : pf ] ) with @xmath226 samples , and with stratified    sampling @xcite .
the filter - smoother estimate ( proposition    [ prop : particle - proper ] ) was used .    in all the mcmc methods , the adaptive random walk metropolis algorithm of @xcite with gaussian proposal distribution was used . the target mean acceptance rate
was set to 0.234 . in the delayed acceptance ,
the first - stage acceptance rate was optimised , and in the pseudo - marginal , the total acceptance rate was optimised . in all cases ,
the adaptive phase was terminated after the burn - in phase .
all the experiments were conducted using the ` bssm ` @xcite package in ` r ` @xcite , which will be made available .
we also experimented with the following algorithmic modifications :    global approximation : :    in the standard version , a laplace approximation is constructed for    every new proposed value of @xmath139 .
we experimented also    faster but less accurate alternative , where we consider only one    series of pseudo - observations constructed at the maximum likelihood    estimate of @xmath139 .
parallel computation : :    we tested our implementation with different number of cores .
the    number of cores used in the experiment is indicated in parenthesis    such as is2 ( 8) .
sub - sampling for memory constraints : :    depending on the problem , we might be interested in either computing    some given summary statistics of @xmath141 , which can    be performed recursively , or constructing a weighted sample from    @xmath436 . in order to save in    storage , we used sub - sampling as in proposition    [ prop : randomise - proper ] for the latter .    our main aim is to compare the performance of the da - mcmc with our is type correction schemes , using the same approximate marginal target distribution in both .
note that , if parallel aspects are not taken into account , the is2 estimator has typically lower cost than da with same parameter @xmath226 . indeed ,
if the second - stage proposal would always be accepted , then da would require approximately the same number of computations as is2
. on the other hand , is1 has typically higher cost than da . because da usually has non - zero second - stage acceptance probability , da with @xmath226 samples has roughly the same cost as is1 using @xmath437 samples , where @xmath438 accounts for rejected second - stage proposals . because the noise in the weighting scheme affects the mixing of the mcmc and da , whereas is1 and is2 are based solely on the approximate marginal mcmc chain plus is type correction , the efficiency of the estimators is likely to be problem dependent .
note also that mcmc and da methods require is steps also during the burn - in , whereas is1 , is2 and is - pf do not .
we consider first the following special case of the model discussed in section [ sec : lin - gauss - ssms ] : @xmath439 with @xmath440 . for testing our algorithms
, we simulated @xmath441 observations from this model with @xmath442 fixed to @xmath443 , and @xmath444 fixed to @xmath445 .    based on our pilot experiments , the pseudo - marginal type mcmc performed well with this model using global marginal likelihood approximation , so we employed it for the experiments .
we also observed that the number of importance samples @xmath226 could be set relatively low without affecting the mixing or accuracy much .
we ran amcmc , mcmc , da , is1 , and is2 algorithms with @xmath446 , using 20,000 mcmc iterations with the first half discarded as burn - in .
we used a uniform prior @xmath447 for the standard deviation parameters , where the cut - off parameter @xmath448 were set to @xmath449 based on the sample standard deviation of @xmath450 , where zeros were replaced with 0.1 .
results were not sensitive to this upper bound .
we performed two experiments . in the first
, we stored only one realisation of @xmath141 at each iteration , and the resulting samples was used to calculate the posterior mean and variance estimates of @xmath139 , @xmath451 , and @xmath452 . in the second experiment , we used all realisations from the simulation smoother and computed the posterior mean and variance estimates of @xmath451 and @xmath452 recursively .
tables [ table : poisson ] and [ table : poisson_summary ] summarise the results over 1000 repeated experiments as discussed above . in table
[ table : poisson ] , the approximate method gave slightly biased estimates for all variables , but with much less cost than exact mcmc .
delayed acceptance decreased the computation costs by about two thirds , while the is2 ( with same @xmath226 and no parallelisation ) almost halves the costs of da . parallel computing with eight cores further decreased the computation time of is - corrected methods , but because of small @xmath226 , the speed - up was limited , as it is impossible to get much lower than with amcmc . using @xmath226 proportional to the block size ( is1 ) does not seem to bring significant benefits in this experiment .
overall , the main differences in da and is corrected methods are related to computational time .
the parameter estimates and their standard errors are comparable .
the pseudo - marginal mcmc provided perhaps slightly smaller standard errors , but due to the much higher computational time , the da and is2 appear preferable in this scenario
.    .means and the corresponding standard errors from 1000 mcmc runs for poisson model with 10 runs of simulation smoother per iteration , with one stored sample path . [ cols=">,>,>,>,>,>,>",options="header " , ]      as an illustration of the estimators in proposition [ prop : particle - smooth - confidence ] , we simulated one realisation @xmath441 from a linear - gaussian ssm of the form @xmath453 with @xmath454 and @xmath455 . given @xmath441 , we estimated @xmath139 via maximum likelihood , resulting @xmath456 and calculated the exact smoothing means @xmath457 $ ] for @xmath458 .
we performed particle smoothing using the filter - smoother ( fs ) and forward - backward smoother ( fbs ) ( proposition [ prop : particle - proper ] and , respectively ) , and repeated these computations @xmath459 times , from which we obtained 95% confidence intervals as in proposition [ prop : particle - smooth - confidence ] .
figure [ fig : ci_plot ] shows confidence intervals from both methods with varying number of particles . for fbs , we used @xmath460 , and for fs , the corresponding squares @xmath461 , making the computational times roughly comparable .
we note that in our implementation , fbs was about 1.4 times slower than fs on average .
it appears that in this experiment , fs outperformed fbs with comparable computational cost .
$ ] with @xmath462 , using forward - backward smoother algorithm ( fbs ) , and filter - smoother algorithm ( fs ) with varying number of particles ( note that algorithms use different amount of particles ) .
red line corresponds to the exact value computed by kalman smoothing . ]    because fbs may be of interest in some other scenarios , we attempted to optimise the choice of @xmath226 for the fbs .
we ran the fbs with varying choices of @xmath226 as above , and using @xmath463 , respectively .
figure [ fig : fbs_time ] shows the square root of average variance estimates @xmath306 of proposition [ prop : particle - smooth - confidence ] over time points @xmath160 , multiplied by the square root of computation time in minutes for different @xmath226 . in this experiment , @xmath464 or @xmath465 appeared good choices , but more importantly , the experiment suggests that the choice of @xmath226 in fbs may be a non - trivial optimisation problem .    .
we suggested to use mcmc targeting a marginal of the posterior distribution and consequent importance sampling type estimation in order to perform easily parallelisable consistent inference over the full posterior distribution .
we focused on state space models ( ssms ) , where importance sampling and particle filters provide natural and efficient correction mechanisms .
our specific application was based on a laplace approximation of ssms with linear - gaussian state dynamics , but arbitrary observation distributions .
our experiments are promising , and we believe that our methodology has potential to make exact inference feasible with a large class of models where only approximate inference is currently possible .
alternative approximation methods in the state space context include simple linearisation - based methods such as the extended kalman filter , moment matching , or variational approximations ; see the recent monograph @xcite .
gaussian latent variable models beyond the state space context could be handled with the laplace approximations , and more refined approaches such as the integrated nested laplace approximation @xcite could provide a useful approximate marginal .
we note that , unlike with purely approximate inference , the approximate posterior @xmath37 does not need to be very accurate .
we suggested importance ( is ) sampling type correction with sequential is and sequential monte carlo ( smc ) .
we note that various smc variations , such as rao - blackwellised smc , and alternative resampling strategies , apply directly @xcite .
randomised sequential quasi - monte carlo @xcite can be useful , when applicable .
smc samplers can be useful beyond the state space context @xcite .
the correction could be based on exact sampling for diffusions @xcite , and debiasing tricks and multilevel monte carlo @xcite can be useful for instance when the true target distribution is based on a continuum model , and only discretisations are accessible .
importance sampling is known to be generally hard to implement efficiently in higher dimensions , and our setting is no exception .
the fact that the approximation @xmath37 is required only on a marginal distribution may help , if the model admits a natural smaller dimensional parameter space , as our state space application .
indeed , in the state space context our approach relies on similar assumptions which make particle mcmc @xcite appealing , with the added assumption of the existence of suitable marginal approximation @xmath37 .
if such approximation is available , we suggest to take advantage of it , as an alternative to particle mcmc .    one potential issue of importance sampling correction , in general , is when the importance weight @xmath46 is unbounded . in our experiments
, we did not observe large values of importance weights , but this is likely due to the fact that our space @xmath466 was bounded . in case @xmath0
is unbounded , it may be that the ratio @xmath467 is unbounded , even if @xmath88 is a reasonable approximation in the ` centre ' of @xmath0 . because bounded @xmath468 is generally desirable ,
we suggest tempering as a possible remedy .
that is , if @xmath37 is our original approximate marginal , the chain @xmath34 may be designed to target @xmath469 instead , where @xmath470 is a tempering parameter .
it may be of practical interest to construct the marginal chain @xmath34 using adaptive mcmc techniques such as @xcite or as discussed in the review @xcite .
we note that our theory does not apply directly with adaptive mcmc , unless the adaptation is stopped after suitable burn - in , like we do in our experiments .
an obvious alternative to our approach is that , instead of running one markov chain targeting @xmath37 and then calculating conditional importance sampling type estimates , one may consider running multiple independent pseudo - marginal type mcmc methods in parallel .
we note that while valid , this approach may suffer from bias if the mcmc chains are not run long enough , and burn - in ( and possible adaptation ) of each chain could be seen wasteful .
it is well - known that pseudo - marginal methods may be slow mixing when the importance weights have large variation  in the extreme case they may lose desirable rates of convergence such as geometric ergodicity @xcite .
large variation of importance weights affects also our method , but we believe that mcmc targeting the approximate target @xmath37 may be often easier to design in an efficient manner .
the authors were supported by an academy of finland research fellowship ( grants 274740 and 284513 ) .
we thank christophe andrieu for insightful remarks .
throughout this section , suppose that @xmath471 is a markov transition probability on @xmath0 and @xmath472 is a markov transition probability from @xmath0 to a space @xmath52 .
we consider here properties of an augmented markov transition probability @xmath473 defined on @xmath58 as follows : @xmath474 we first state the following basic result .
[ lem : aug - prop ] the properties of @xmath471 and the augmented chain @xmath473 are related as follows :    a.   [ lem : aug - mim - irr ] let @xmath475 denote the set of @xmath476-irreducibility measures of a markov transition probability @xmath471 , then @xmath477 b.   [ lem : aug - mim - mim ] the implications in hold when @xmath475 and @xmath478 are replaced with sets of maximal irreducibility measures of @xmath471 and @xmath473 , respectively . c.   [ item : aug - invar ] the invariant probabilities of @xmath471 and @xmath473 satisfy : @xmath479 the above implications hold also with invariance replaced by reversibility .
d.   [ item : aug - harris ] @xmath471 is harris recurrent if and only if @xmath473 is harris recurrent .
e.   [ item : aug - iterates ] suppose @xmath480 is measurable and such that @xmath481 and @xmath482 are well - defined
. then , for any @xmath483 , @xmath484    the inheritance of irreducibility measures ( [ lem : aug - mim - irr ] ) , maximal irreducibility measures ( [ lem : aug - mim - mim ] ) , invariant measures , and reversibility are straightforward .    for harris recurrence ,
let the probability @xmath485 be a maximal irreducibility measure for @xmath471 , then @xmath486 is the maximal irreducibility measure of @xmath473 .
let @xmath487 with @xmath488 , and choose @xmath489 such that @xmath490 , where @xmath491 with @xmath492 .
notice that @xmath493 where @xmath494 are the hitting times of @xmath89 to @xmath495 .
this concludes the proof because @xmath496 are independent bernoulli random variables with success probability at least @xmath497 .
the converse statement is similar .    for
, it is enough to notice that for any @xmath498 and @xmath483 , it holds that @xmath499 .
we next state the following generic result which gives an expression for the asymptotic variance of an augmented markov chain .
[ prop : aug - asvar ] let @xmath500 and define the conditional mean @xmath501 and the conditional variance @xmath502 , then @xmath503 whenever @xmath504 is well - defined .
let @xmath267 be the stationary markov chain with transition probability @xmath473 .
@xmath505\bigg ) \\      & = \check{\mu}(h^2 ) + \frac{2}{n } \sum_{i=1}^{n-1 }      \sum_{\ell=1}^{n - i } { \mathbb{e}}[h(x_0,s_0)h(x_\ell , s_\ell)],\end{aligned}\ ] ] by stationarity .
we may write by lemma [ lem : aug - prop ] , @xmath506      & = \int \check{\mu}({\mathrm{d}}x\times { \mathrm{d}}s ) h(x , s ) ( k^\ell m_h)(x ) \\      & = { \mathbb{e}}[m_h(x_0 ) m_h(x_\ell)].\end{aligned}\ ] ] we deduce that @xmath507 where @xmath508 .
the claim follows by taking limit @xmath305 .
we finally state a simple result about the relationship of the solutions of the poisson equation for augmented chains , used in appendix [ app : jump ] .
[ lem : aug - poisson ] suppose that @xmath500 .
if there exists @xmath509 such that @xmath510 , where @xmath511 , then @xmath512    it is clear that @xmath513 and @xmath514 , so @xmath515
we follow @xcite and prove the statement assuming that the resampling scheme @xmath516 satisfies the following property : @xmath517 for all @xmath518 .
we can do this without loss of generality , because this condition would be satisfied if we applied an independent random permutation of indices to the resampling scheme satisfying assumption [ a : resampling ] .
the statement is valid without such randomisation , because both the algorithm and the statements are independent of ordering .    for , define the filtrations @xmath519 and @xmath520 , and functions @xmath521 , and for @xmath522 @xmath523 clearly @xmath524 , and all @xmath525 are ( almost everywhere ) well - defined if the latter integral is well - defined .
let us first observe that for @xmath147 , @xmath526      & =      { \mathbb{e}}\big [ { \mathbb{e}}\big [      \omega_t^{(i ) }      f_t(\bar{\alpha}_{t-1}^{(a_{t-1}^{(i)})},\alpha_t^{(i ) } )      { \mathrel{\big|}}{\mathcal{g}}_{t-1 }      \big ]      { \mathrel{\big|}}{\mathcal{f}}_{t-1 }      \big ]       \label{eq : unbiased - step }       \\      & =      { \mathbb{e}}\big [      f_{t-1}(\bar{\alpha}_{t-1}^{(a_{t-1}^{(i ) } ) } )      { \mathrel{\big|}}{\mathcal{f}}_{t-1 }      \big ]       \notag      \\      & =      \sum_{i=1}^m \bar{\omega}_{t-1}^{(i ) }      f_{t-1}(\bar{\alpha}_{t-1}^{(i ) } ) .
\notag\end{aligned}\ ] ] by applying recursively , we obtain @xmath527      & = { \mathbb{e}}\bigg[\bigg(\prod_{t=1}^{t-1 } \frac{1}{m}\omega_t^ * \bigg )       \frac{1}{m }      \sum_{i=1}^m       { \mathbb{e}}\big [ \omega_t^ *   \bar{\omega}_t^{(i ) }      f_t(\bar{\alpha}_{t-1}^{(a_{t-1}^{(i)})},\alpha_t^{(i)}){\mathrel{\big|}}{\mathcal{f}}_{t-1}\big]\bigg ] \\ & = { \mathbb{e}}\bigg[\bigg(\prod_{t=1}^{t-1 } \frac{1}{m } \omega_t^*\bigg ) \sum_{i=1}^m       \bar{\omega}_{t-1}^{(i ) } f_{t-1}(\bar{\alpha}_{t-1}^{(i ) } )     \bigg ] \\     & = \frac{1}{m}\sum_{i=1}^m          { \mathbb{e}}\big [ \omega_1^ * \bar{\omega}_t^{(i ) }      f_1(\alpha_1^{(i)})\big],\end{aligned}\ ] ] which equals to @xmath528 by a similar calculation as in .
the statement is equivalent with @xmath529 = \int p^{(\theta)}(\alpha_{1:t},y_{1:t } ) f(\alpha_{1:t } )      { \mathrm{d}}\alpha_{1:t}.      \label{eq : all - trajectories - proper}\ ] ] similarly as above , we have for @xmath522 , @xmath530 \\      & = \int \sum_{j=1}^n \bar{\omega}_{t-1}^{(j ) } \mu_t(\alpha_t\mid      \alpha_{t-1}^{(j ) } ) g_t(y_t\mid \alpha_t )      \frac {        \bar{\omega}_{t-1}^{(i_{t-1 } ) }        \mu_t(\alpha_t\mid \alpha_{t-1}^{(i_{t-1 } ) } )         } {    \sum_{\ell=1}^m       \bar{\omega}_{t-1}^{(\ell ) }       \mu_t(\alpha_t\mid\alpha_{t-1}^{(\ell ) } ) }   f_t(\alpha_{1:t}^{(i_{1:t } ) } ) { \mathrm{d}}\alpha_t \\ & = \bar{\omega}_{t-1}^{(i_{t-1 } ) } f_{t-1}(\alpha_{1:t-1}^{(i_{1:t-1})}),\end{aligned}\ ] ] and application of this recursively leads to .
the result follows also from , by first summing over the indices @xmath531 .
the last result follows as a special case of .
consider the artificial trivial markov chain @xmath89 with @xmath532 for @xmath102 which is harris ergodic with respect to @xmath533 , with @xmath534 , and let @xmath535 .
then , @xmath536 and @xmath537 with @xmath538 forms a proper weighting scheme for @xmath539 .
the strong consistency follows from proposition [ prop : proper - consistency ] and the central limit theorem from proposition [ prop : proper - clt ] , because @xmath34 is trivially uniformly ergodic and @xmath540 .
note that @xmath541 implying @xmath542 .
similarly , the convergence of @xmath543 follows from proposition [ prop : importance - var ] .
let @xmath544 $ ] , then @xmath545 where @xmath546 $ ] .
this implies that @xmath547    whenever @xmath548 , we may write @xmath354 = \frac{n^{-1/2 }        \sum_{k=1}^n w_k \sum_{i=1}^n v_k^{(i ) }        \bar{f}(x_k , z_k^{(i)})}{n^{-1 }        \sum_{j=1}^n w_j}.      \label{eq : clt - form}\ ] ] the denominator converges to @xmath42 almost surely , so by slutsky s lemma it is enough to show that the numerator converges in distribution to @xmath549 .
this follows from proposition [ prop : aug - clt ] , because @xmath291 .    for @xmath345 large enough such that @xmath550 for @xmath551 ,
we may write @xmath552 the denominator converges to @xmath553 , and the numerator can be written as @xmath554,\ ] ] and @xmath555 . the term @xmath556 , and because @xmath557 , the remainder terms @xmath558 and @xmath559 , as @xmath560 .
the following proposition complements ( * ? ? ? * lemma 1 ) and @xcite , which are stated for more specific cases .
[ prop : jump - properties ] suppose @xmath89 is a markov chain with non - degenerate transition probability @xmath471 and @xmath322 its jump chain with corresponding holding times @xmath561 ( definition [ def : jump ] ) .
then , the following hold :    a.   [ item : jump - trans ] @xmath322 is a markov chain with transition probability @xmath562 b.   [ item : jump - joint - trans ] the holding times @xmath561 are conditionally independent given @xmath322 , and each @xmath333 has geometric distribution with parameter @xmath563 . c.   [ item : jump - invar ] if @xmath471 admits invariant probability @xmath564 , then @xmath565 admits the invariant probability @xmath566 in addition , if @xmath471 is reversible with respect to @xmath5 , then @xmath565 is reversible with respect to @xmath567 . d.   [ item : psi - irreducibility ] @xmath89 is @xmath568-irreducible if and only if @xmath322 is @xmath568-irreducible , with the same maximal irreducibility measure .
e.   [ item : harris - recurrence ] @xmath89 is harris recurrent if and only if @xmath322 is harris recurrent .
the expression of the transition probability is due to straightforward conditioning , and was observed in @xcite .
the invariance follows from @xmath569 \\                & = \frac{1}{\mu(a)}\bigg [ \mu(a ) -
\int_a \mu({\mathrm{d}}x)\big(1-a(x)\big)\bigg ]                = \tilde{\mu}(a ) ,            \end{aligned}\ ] ] and the reversibility is shown in @xcite .
for it is sufficient to observe that @xmath570 which holds because the sets @xmath571 and @xmath572 coincide .
similarly , holds because @xmath573 where @xmath574 and @xmath575 .          from the expression of the jump chain transition probability in proposition [ prop : jump - properties ]
, we obtain for any @xmath582 with @xmath583 well - defined , @xmath584 consequently , we observe that @xmath585 implying . because @xmath579 , lemma [ lem : poisson - variance ] and a straightforward calculation yield @xmath586 ,            \end{aligned}\ ] ] where @xmath587 .
similarly , by lemma [ lem : poisson - variance ] , @xmath588 which allows us to conclude .
[ lem : poisson - variance ] suppose the markov transition probability @xmath471 on @xmath0 is harris ergodic with respect to @xmath5 and @xmath589 . if there exists @xmath509 which solves the poisson equation , @xmath590 then @xmath591 and the markov chain @xmath34 following @xmath471 satisfies a clt .
let @xmath14 , @xmath592 and @xmath266 be the markov transition probabilities of @xmath34 , @xmath313 and @xmath593 , respectively , and let @xmath37 , @xmath594 and @xmath268 be the corresponding invariant probabilities .
because @xmath34 and is @xmath288-geometric and @xmath595 , there exists a solution of poisson s equation @xmath596 such that @xmath597 .
geometric ergodicity of @xmath34 implies also @xmath598 for @xmath37-almost every @xmath64 @xcite , so @xmath599 .
proposition [ prop : jump - variance ] implies that @xmath600 , and @xmath601 .
denote then @xmath602 which has conditional mean @xmath603 , and @xmath604 is finite because @xmath605 \le \frac{2-\alpha(x)}{\alpha^2(x ) }      m_\xi^{(2)}(x).\ ] ] lemma [ lem : aug - poisson ] therefore implies that there exists @xmath606 such that @xmath607 , and consequently lemma [ lem : poisson - variance ] implies that @xmath593 satisfies a clt for @xmath608 .    for the expression of the asymptotic variance , proposition [ prop : aug - asvar ] applied with @xmath609 , @xmath610 , @xmath611 and @xmath612
, implies @xmath613 where , by the variance decomposition formula , @xmath614{\mathrel{\big|}}\tilde{x}_k = x\big ) \\      &
= v_{n\xi}(x ) + w_u^2(x ) \bar{f}^{*2}(x )      \big(1-\alpha(x)\big)/\alpha^2(x).\end{aligned}\ ] ] proposition [ prop : jump - variance ] implies that @xmath615.\ ] ] if we write the scaled estimator analogously as in , we notice that the clt holds for the numerator , with asymptotic variance @xmath616/\pi_a(\alpha).\ ] ] the denominator converges to @xmath617 , so we conclude with slutsky s lemma .
a.  beskos , o.  papaspiliopoulos , g.  o. roberts , and p.  fearnhead .
exact and computationally efficient likelihood - based estimation for discretely observed diffusion processes .
, 68(3):333382 , 2006 .
( with discussion ) .