an interesting framework for studying the non - perturbative quantum field theories is through the use of anti - de sitter / conformal field theory ( ads / cft ) correspondence @xcite . according to this duality , in principle
, one can perform gravity calculations to find information about the field theory side or vice versa . in this context , the central charges of the dual theory ( cft ) relate to coupling constants of its dual gravity .
therefore , einstein gravity with one coupling constant restricts the dual theory to a limited class of cft with equal central charges @xcite . for extension of the duality beyond this limit
, one needs to involve higher curvature terms in the gravity action .
it is clear that each correction term introduces a new coupling constant and therefore one may have cft theory with different central charges . indeed , this procedure leads to the richness of the cft theory @xcite .
the most natural extension of general relativity with higher curvature terms and with the assumption of einstein  that the left hand side of the field equations is the most general symmetric conserved tensor containing no more than two - derivatives of the metric  is lovelock theory .
lovelock @xcite found the most general symmetric conserved tensor satisfying this property .
the resultant tensor is nonlinear in the riemann tensor and differs from the einstein tensor only if the spacetime has more than 4 dimensions .
although lovelock gravity leads to second - order field equations and it has ghost free ads solution @xcite , it has been recently shown that quadratic and cubic gravities entail causality violation and there are stringent conditions on the coupling constants @xcite .
the problem with the total action of einstein gravity is that it is divergent when evaluated on the solutions @xcite . due to
this fact , all the other conserved quantities which is calculated through the use of this action is also divergent .
one way of eliminating these divergences is through the use of background subtraction method of brown and york @xcite . in this method ,
the boundary surface is embedded in another ( background ) spacetime , and one subtracts the action evaluated on the embedded surface of the background spacetime from the total action .
such a procedure causes the resulting physical quantities to depend on the choice of reference background .
furthermore , it is not possible in general to embed the boundary surface into a background spacetime . for asymptotically ads solutions of einstein gravity ,
one may remove the non - logarithmic divergences in the action by adding a counterterm action which is a functional of the boundary curvature invariants @xcite .
indeed , this counterterm method furnishes a means for calculating the action and conserved quantities intrinsically without reliance on any reference spacetime @xcite .
although there may exist a very large number of possible invariants , only a finite number of them are non - vanishing in a given dimension on a boundary at infinity .
this method has been applied to many cases such as black holes with rotation , nut charge , various topologies , rotating black strings with zero curvature horizons and rotating higher genus black branes @xcite .
although the counterterm method applies for the case of a specially infinite boundary , it was also employed for the computation of the conserved and thermodynamic quantities in the case of a finite boundary @xcite .
all of the works mentioned in the previous paragraph were limited to einstein gravity .
although the counterterm of lovelock gravity with flat horizon has been introduced @xcite , only a few works related to the counterterm method have been done for lovelock gravity with curved horizon .
this is due to the fact that even for einstein gravity , the systematic construction that provides the form of the counterterms becomes cumbersome for high enough dimensions @xcite . indeed in this method , one should reconstruct the spacetime metric by solving iteratively the field equations in the fefferman - graham frame @xcite : @xmath0 dx^{i}dx^{j } , \label{fefmet}\]]where @xmath1 is the boundary data of an initial - value problem governed by the equations of motion .
however , even for einstein - hilbert theory , solving the coefficients @xmath2 in eq .
( [ fefmet ] ) as covariant functionals of @xmath3 is only possible for low enough dimensions .
thus , it is expected that the holographic renormalization procedure would be even more complicated in lovelock gravity because of the nonlinearity of field equations . indeed , because of the nonlinearity of the field equations solving @xmath4 in eq .
( [ fefmet ] ) as covariant functionals of @xmath5 would be even more cumbersome .
so , the authors in ref .
@xcite presented an alternative construction of kounterterms . instead of adding counterterms to cancel the divergence at the boundary explained above , they circumvented the difficulties of the standard method by using kounterterms which depend on the intrinsic and the extrinsic curvatures of the boundary .
they selected the kounterterms as the boundary terms which are regular on the asymptotic region .
indeed , the regularization process is encoded in the boundary terms already presented and there is no need to add further counterterms .
the exact rotating solutions of lovelock gravity with curved horizon are not introduced till now .
indeed , only static solutions of lovelock gravity with different matter fields are known @xcite .
so , because of the difficulties of the holographic renormalization procedure in lovelock gravity and the nonexistance of an exact rotating solution of this theory , we limit ourselves to the case of counterterms of lovelock gravity for static solutions .
the counterterms of asymptotically ads static solutions of gauss - bonnet gravity have been introduced in ref . @xcite .
also , the finite action and global charges of asymptotically de sitter static solutions has been obtained in ref .
@xcite .
here we like to apply the counterterm method to the case of the static solutions of the field equations of third order lovelock gravity with curved horizon .
we define the cosmological constant in such a way that the maximally symmetric ads spacetime @xmath6be the vacuum solution of lovelock gravity .
( [ adsmet ] ) @xmath7 is the metric of an ( @xmath8)-dimensional maximally symmetric space with curvature constant @xmath9 and volume @xmath10 . indeed , this choice of cosmological constant makes the asymptotic form of the solutions of lovelock gravity to be exactly the same as that of einstein gravity .
thus , we expect that the counterterm introduced for einstein gravity in @xcite may remove the power law divergences in the action of lovelock gravity . although the counterterms which should be added to gauss - bonnet gravity in order to remove the power law divergences of the action for static solutions are introduced in ref .
@xcite , they depend on the gauss - bonnet coefficient . however , because of our choice of the cosmological constant , our counterterms are the same as those of einstein gravity and are independent of lovelock coefficients . in order to check our counterterms , we calculate the finite action and the mass of the black hole through the use of counterterm method
. then , we use these finite quantities and the gibbs - duhem relation to obtain the entropy .
we find that the calculated entropy of the black holes is consistent with the wald s formula @xcite . as another test of our counterterm method
, we show that the mass obtained through the use of counterterm method satisfies the first law of thermodynamics .
we , also , perform a stability analysis of the black hole solutions in canonical ensemble and investigate the effects of third order lovelock term on the stability .
this paper is organized as follows . in section [ sol ]
, we review the well - defined action of lovelock gravity . in section [ count ] , we introduce the counterterms for third order lovelock gravity for static spacetimes . we also , introduce the finite stress energy tensor of this theory .
section [ therm ] is devoted to the thermodynamics of the black hole solutions of the theory .
we calculate the finite action , the total mass , the temperature , the charge and the electric potential .
we calculate the entropy through the use of gibbs - duhem relation and wald formula and find that they are consistent .
we , also , investigate the first law of thermodynamics . in sec .
[ stab ] , we investigate the thermal stability of the solutions in canonical ensemble .
we finish our paper with some concluding remarks .
the bulk action of lovelock gravity in @xmath11 dimensions may be written as @xcite @xmath12 } \alpha _ { p}\left ( \mathcal{l}_{p}-2\lambda _ { p}\right ) + i_{mat }   \label{action1}\]]with @xmath13 $ ] denoting the integer part of @xmath14 , @xmath15 s ( @xmath16 ) are lovelock coefficients , @xmath17is the euler density of a @xmath18-dimensional manifold , @xmath19 is the general asymmetric kronecker delta and@xmath20the action ( [ action1 ] ) is written in such a way that the maximally symmetric ads spacetime ( [ adsmet ] ) is the vacuum solution of action ( action1 ) . in this notation ,
the independent coupling constants are @xmath21 and all the lovelock coefficients .    from a geometric point of view
the lagrangian of the action ( [ action1 ] ) in @xmath22 + 1 $ ] and @xmath22 + 2 $ ] dimensions is the most general lagrangian that yields second order field equations , as in the case of einstein - hilbert action which is the most general lagrangian producing second order field equations in three and four dimensions . in the rest of the paper , we work in a unit system with @xmath23 and the dimensionless lovelock coefficients @xmath24 defined as@xmath25with the definition ( [ alphahat ] ) , the cosmological constant for ads spacetime is@xmath26 } ( -1)^{p}\tilde{% \alpha}_{p}.\ ] ]    in this paper , we consider the third order lovelock gravity in the presence of electromagnetic field .
thus , the action of matter field is @xmath27}\partial ^{\lbrack \mu } a^{\nu ] } , \]]where @xmath28 is the electromagnetic potential .
the first term in lovelock lagrangian is the einstein - hilbert term @xmath29 , the second term is the gauss - bonnet lagrangian @xmath30 , the third term is @xmath31and the cosmological constant is@xmath32    as in the case of einstein - hilbert action , the action ( [ action1 ] ) does not have a well - defined variational principle , since one encounters a total derivative that produces a surface integral involving the derivatives of @xmath33 normal to the boundary @xmath34
. these normal derivatives of @xmath35 can be canceled by the variation of the surface action @xcite @xmath36 } \alpha _ { p}q_{p},\]]where @xmath37}^{[jj_{1}\cdots j_{2p-1}]}\,k_{j_{1}}^{i_{1}}\,\times   \notag \\ & & \times \left ( \frac{1}{2}\,\hat{r}_{j_{2}j_{3}}^{i_{2}i_{3}}(\gamma ) -t^{2}k_{j_{2}}^{i_{2}}k_{j_{3}}^{i_{3}}\right ) \cdots \left ( \frac{1}{2}\,% \hat{r}_{j_{2p-2}j_{2p-1}}^{i_{2p-2}i_{2p-1}}(\gamma ) -t^{2}\,k_{j_{2p-2}}^{i_{2p-2}}k_{j_{2p-1}}^{i_{2p-1}}\right ) . \label{surlag}\end{aligned}\]]in eq .
( [ surlag ] ) @xmath38 and @xmath39 are the induced metric and extrinsic curvature of the boundary @xmath34 , respectively .
the explicit form of the first three terms of eq .
( [ surlag ] ) are @xcite : @xmath40where @xmath41 and @xmath42 are the traces of @xmath43and @xmath44k_{ab } \notag \\ & & -(4k^{3}-12kk_{ed}k^{ed}+8k_{de}k_{f}^{e}k^{fd})k_{ac}k_{b}^{c}-24kk_{ac}k^{cd}k_{de}k_{b}^{e } \notag \\ & & + ( 12k^{2}-12k_{ef}k^{ef})k_{ac}k^{cd}k_{db}+24k_{ac}k^{cd}k_{de}k^{ef}k_{bf}\ } , \label{pab}\end{aligned}\]]respectively . in eq .
( [ isurf ] ) @xmath45 is the einstein tensor , @xmath46 is the intrinsic curvature and @xmath47 is the gauss - bonnet tensor of the metric @xmath38 given as @xmath48
it is well known that the action @xmath49 is not finite for asymptotically ads solutions .
inspired by ads / cft correspondence , one needs to add counterterms to the gravity action in order to get a finite action .
these counterterms are made from the curvature invariants of the boundary metric with the coefficients of the higher curvature terms chosen so that power law divergences in the bulk are canceled for all possible boundary topologies permitted by the equations of motion . at any given dimension
there are only a finite number of counterterms that do not vanish at infinity .
this does not depend upon what the gravity theory is  i.e. whether or not it is einstein , gauss - bonnet , 3rd order lovelock , etc . indeed , for asymptotically ads solutions
, the boundary counterterms that cancel the divergences in einstein gravity may also cancel the divergences in lovelock gravity if one chooses the cosmological constant as in eq .
this is due to the fact that the @xmath50th order lovelock lagrangian @xmath51 calculated for the metric ( [ adsmet ] ) is independent of lovelock coefficients .
that is , the different orders of lovelock action do not mix with each other and one may find the counterterms for different orders of lovelock terms separately .
this point makes the calculation easier . of course
, the coefficients of the various counterterms for different lovelock terms will be different , depend only on @xmath21 and will be independent of lovelock coefficients . thus , using the counterterms of einstein gravity @xcite , we may write the counterterms of third order lovelock gravity as @xmath52where @xmath53the coefficients @xmath54 , @xmath55 , @xmath56 and @xmath57 in eq .
( [ ict - p ] ) should depend on @xmath21 .
these coefficients for einstein gravity ( @xmath58 , @xmath59 ) are @xcite @xmath60one may note that the coefficients @xmath54 , @xmath55 , @xmath56 and @xmath57 are independent of lovelock coefficients .
we apply the counterterms ( [ ict - p ] ) to various static solutions of gauss - bonnet and third order lovelock gravity with different topolgy and find that these coefficients are @xmath61and @xmath62for gauss - bonnet and third order lovelock gravity , respectively .
the reason that we use exactly the counterterms of einstein gravity is as follows .
first , the boundary at @xmath63 for static solutions is a constant curvature hypersurface and therefore no six - derivative term will be appeared in the counterterms . in other words , all the terms of a specific order of counterterms [ for example @xmath64 and @xmath65 in @xmath66 ) $ ] ] for static solutions
are proportional to @xmath67 . also , @xmath68 is proportional to @xmath67 .
therefore , in order to remove the divergences of the action which are proportional to @xmath69 , any combination of @xmath64 , @xmath65 and @xmath68 can be used .
so , we just use exactly the counterterms of einstein gravity .
second , @xmath54 * * , * * @xmath55 * * , * * @xmath56 * * , * * @xmath57 , for @xmath70 and @xmath71 are proportional to those of einstein counterterm independent of the dimensions .
that is , the dimensional - dependence of these coefficients are the same as those in einstein gravity .
third , as we will see in the next section , the entropy calculated through the use of gibbs - duhem relation and the mass and action calculated by our counterterms is consistent with the entropy obtained by use of wald s formula .
fourth , the mass calculated by our counterterms satisfies the first law of thermodynamics .
while the total action @xmath72 is appropriate in grand - canonical ensemble where @xmath73 is zero at the boundary , the appropriate action in the canonical ensemble where the electric charge is fixed is @xcite @xmath74thus both in canonical and grand - canonical ensemble , the variation of total action about the solutions of the field equations is @xmath75so , the energy - momentum tensor can be written as : @xmath76the explicit expressions of @xmath77 and @xmath78 are somewhat cumbersome so we give them in the appendix .    to compute the conserved charges of the spacetime , we choose a spacelike hypersurface @xmath79 in @xmath34 with metric @xmath80 , and write the boundary metric in adm form : @xmath81where the coordinates @xmath82 are the angular variables parameterizing the hypersurface of constant @xmath83 around the origin , and @xmath84 and @xmath85 are the lapse and shift functions , respectively . when there is a killing vector field @xmath86 on the boundary , then the quasilocal conserved quantities associated with the stress tensors of eq .
( [ stres ] ) can be written as @xmath87where @xmath88 is the determinant of the metric @xmath89 , and @xmath90 is the timelike unit normal vector to the boundary @xmath79 . in the context of counterterm method , the limit in which the boundary @xmath79 becomes infinite ( @xmath91 )
is taken , and the counterterm prescription ensures that the action and conserved charges are finite .
no embedding of the surface @xmath79 into a reference of spacetime is required and the quantities which are computed are intrinsic to the spacetimes .
the field equation of third order lovelock gravity for the static metric @xmath92 in the presence of electromagnetic field may be written as @xmath93-\tilde{\alpha}_{2}[\psi ^{2}(r)-1]+[\psi ( r)-1]+\frac{ml^{2}}{r^{n}}-\frac{2}{(n-1)(n-2)}\frac{q^{2}l^{2}}{r^{2(n-1)}}% = 0 ,   \label{cubic}\]]where @xmath94l^{2}/r^{2}$ ] , @xmath95 is the charge of the black hole and @xmath96 is the integration constant which is related to the mass of the solution . the electromagnetic potential for the above metric is@xmath97the mass parameter @xmath96 may be written in terms of horizon radius and @xmath95 by using the fact that @xmath98 , and therefore @xmath99where@xmath100as one expects , @xmath101 ( @xmath102 ) is the root of eq .
( [ cubic ] ) for the vacuum spacetime ( @xmath103 and @xmath104 ) .
the three solutions of the cubic equation ( [ cubic ] ) are @xmath105where @xmath106all of the above three roots could be real in the appropriate range of @xmath107 and @xmath108 .
the second and third solutions are real provided @xmath109 . here
, we will consider only the first solution @xmath110 which is real provided @xmath111 at any @xmath83 .
now , we investigate the thermodynamics of the black hole solutions . the temperature of the event horizon may be calculated through the use of analytic continuation of the metric .
one obtains @xmath112n% \frac{r_{+}^{2}}{l^{2}}+k(n\mu -2\eta ) -\frac{2q^{2}}{(n-1)r_{+}^{2(n-2)}}% \right ) ,   \label{temp}\]]where@xmath113the charge of the black holes per unit volume can be calculated by integrating the flux of the electric field as @xmath114the electric potential @xmath115 , measured at infinity with respect to the horizon is defined as @xmath116where @xmath117 is the null generator of the horizon .
one obtains @xmath118    the finite total action in grand - canonical and canonical ensembles can be found through the use of the counterterm method introduced in the last section .
it is a matter of straightforward calculations to show that the total action is finite .
since we are interested in the stability of the solutions in canonical ensemble , we calculate the finite euclidian action per unit volume @xmath10 in this ensemble .
one obtains @xmath119 \frac{r_{+}^{n}}{l^{2}}-k(n\mu -2\eta ) \frac{\xi } { % \eta } r_{+}^{n-2}\right\}+i_{0}\end{aligned}\]]where @xmath120 is the euclidean time period ( the inverse of temperature ) , @xmath121 is given in eq .
( [ eta ] ) , @xmath122 is@xmath123and @xmath124 is @xmath125one should note that @xmath124 appears only in odd dimensions ( even @xmath126 ) .    for our static solution
, there is a killing vector field @xmath127 on the boundary , and therefore the quasilocal conserved quantity associated with the stress tensors of eq .
( [ stres ] ) is the mass of the black hole . using the counterterm method introduced in the last section ,
it is a matter of calculation to obtain the mass of black hole as @xmath128where @xmath129 is the casimir energy for the vacuum ads metric per volume @xmath10 , which is nonzero only in odd dimensions . in order to have positive energy @xmath130 , one should restrict the range of lovelock coefficients in terms of @xmath131 , @xmath21 and @xmath132 as@xmath133that is , the third order lovelock coefficient has a lower limit in terms of @xmath107 , @xmath95 , @xmath21 and @xmath132 .
using the mass and action per unit volume @xmath10 calculated through the use of counterterm method , one may calculate the entropy per unit volume through the use of gibbs - duhem relation @xmath134 as @xmath135it is worth to note that the entropy of the black hole solution per unit volume @xmath10 calculated by the use of gibbs - duhem relation is consistent with the calculation through the use of  wald formula wald , myers2 @xmath136where the integration is done on the @xmath137-dimensional spacelike hypersurface of killing horizon , @xmath138 is the induced metric on it , @xmath139 is the determinant of @xmath138 and @xmath140 is the @xmath141nd order lovelock lagrangian of @xmath142 .
also , one may note that the thermodynamic quantities calculated in this section satisfy the first law of thermodynamics@xmath143
now , we study the thermal stability of the black hole solutions of third order lovelock gravity in canonical ensemble .
first , we investigate the conditions of having black hole solution in lovelock gravity .
the solution given by eq .
( [ sol1 ] ) presents a black hole solution provided @xmath144 has at least one real positive root .
this occurs if @xmath145 , or @xmath146 where @xmath147 and @xmath148 satisfy the following equation : @xmath149n\frac{r_{ext}^{2}}{l^{2}}% + k[(n-2)+k(n-4)\frac{l^{2}}{r_{ext}^{2}}+k^{2}(n-6)\frac{l^{4}}{r_{ext}^{4}}-% \frac{2q_{ext}^{2}}{(n-1)r_{ext}^{2(n-2)}}=0 .
\label{ext}\ ] ] also , the temperature of a physical black hole should be positive .
one may note that the temperature changes its sign at the root of @xmath150 ( @xmath151 ) .
the radius @xmath152 depends on @xmath21 and lovelock coefficients , while @xmath147 depends on @xmath95 too . in fig .
[ fig1 ] , the vertical line is @xmath153 line .
this figure shows that one may have only small ( @xmath154 ) and large black holes , and there is no medium black hole solution ( @xmath155 ) .
this feature does not occur for black holes of einstein gravity or lovelock gravity with positive lovelock coefficients , since @xmath156 .
thus , for the case of lovelock gravity with negative lovelock coefficient(s ) one may divide the black holes into two classes with negative and positive @xmath157 .    in the canonical ensemble @xmath131
is fixed and therefore the black hole solutions are stable provided @xmath158 in the range that @xmath159 is positive . using the expressions for mass and entropy given in eq .
( [ mass ] ) and ( [ ent ] ) , one may calculate @xmath160 : @xmath161where @xmath162 \\ & & -k^{2}r_{+}^2\left[n(\tilde{\alpha}_{2}(15\tilde{\alpha}_{3}+1)-15\tilde{% \alpha}_{3}(1 + \tilde{\alpha}_{3}))-8\tilde{\alpha}_{2}\right]l^2 - 2k\left[n({% \tilde{\alpha}_{2}}^2 - 2\tilde{\alpha}_{3})- 4{\tilde{\alpha}_{2}}^2 - 6\tilde{% \alpha}_{3}\right]l^4 \\ & & -3k^2\tilde{\alpha}_{3}\tilde{\alpha}_{2}r_{+}^{-2}(n-8)l^{6 } -3k{\tilde{% \alpha}_{3}}^2r_{+}^{-4}(n-6)l^8.\end{aligned}\ ] ] .     versus @xmath132 for @xmath163 , @xmath164 , @xmath165 and @xmath166 in einstein gravity ( solid ) and lovelock gravity ( dotted ) with @xmath167 and @xmath168 .
the vertical line is @xmath169 line.,title="fig : " ]     ( dotted ) and @xmath170 ( solid ) versus @xmath132 for @xmath171 and @xmath166 .
the vertical line is @xmath172 line.,title="fig : " ]     ( dotted ) and @xmath173 ( solid ) versus @xmath174 for @xmath175 , @xmath176 , @xmath164 , @xmath165 , @xmath166 and @xmath177 .
the vertical line is @xmath169 line.,title="fig : " ]     ( dotted ) and @xmath178 ( solid ) versus @xmath174 for @xmath175 , @xmath176 , @xmath164 , @xmath165 , @xmath166 and @xmath179.,title="fig : " ]     ( dotted ) and @xmath180 ( solid ) versus @xmath181 for @xmath182 , @xmath183 , @xmath104 , @xmath164 , @xmath165 and @xmath166 .
the vertical line is @xmath169 line.,title="fig : " ]     ( dotted ) and @xmath184 ( solid ) versus @xmath174 for @xmath185 , @xmath176 , @xmath164 , @xmath165 , @xmath166 and @xmath186.,title="fig : " ]     ( dotted ) and @xmath187 ( solid ) versus @xmath174 for @xmath175 , @xmath176 , @xmath164 , @xmath165 , @xmath166 and @xmath177 .
the vertical line is @xmath169 line.,title="fig : " ]    to investigate the stability of black holes of third order lovelock gravity in canonical ensemble , we should find the sign of @xmath188 , when @xmath159 is positive . in order to investigate the stability , we plot @xmath159 and @xmath189  in one figure .
the allowed region for investigation of @xmath190  is when @xmath159 is positive . in fig .
[ fig2 ] , the vertical dotted - line is @xmath150 line .
this figure shows that the small black holes ( @xmath154 ) divided into stable and unstable black holes .
that is a hawking - page phase transition exists for small black holes between very small and small black holes .
there is no phase transition for large black holes ( @xmath191 ) as one may see in fig .
[ fig2 ] . for investigating the effect of third order term of lovelock gravity on the stability of the black hole solutions , we plot @xmath159 and @xmath188 versus @xmath108 for small and large black holes .
figure [ fig3 ] shows that there is no large black holes for large negative @xmath108 , while the large black holes for small negative @xmath108 are not stable . as @xmath192 becomes larger
, large stable black holes may exist .
figure [ fig4 ] shows that there is no small black hole solution for positive @xmath192 , while the small black hole with negative @xmath108 are stable .
figure [ fig5 ] shows that there is no negative-@xmath157 ( small ) uncharged black hole with event horizon , while the large black holes ( @xmath193 ) show a phase transition between large and very large ones .
one may see in fig .
[ fig5 ] that the large black holes with positive @xmath157 show a phase transition , while these black holes with negative @xmath157 are not stable .
finally , figs .
[ fig6 ] and [ fig7 ] show the temperature and @xmath194 versus @xmath108 for small and large uncharged black holes .
figure [ fig6 ] shows that there is no uncharged black hole with negative @xmath157 , while the black holes with positive @xmath157 are stable . on the other hand ,
the large uncharged black holes with positive @xmath157 show a phase transition as @xmath192 becomes larger and these solutions are not stable for very large negative @xmath108 .
the concepts of action plays a central role in gravitation theories , but the sum of the bulk and the surface terms diverges . in this paper , we introduced the counterterms that remove the non - logarithmic divergences of static solutions of third order lovelock gravity .
we did this by defining the cosmological constant in such a way that the ads metric ( [ adsmet ] ) is the vacuum solution of lovelock gravity .
indeed , the cosmological constant ( [ cos ] ) makes the asymptotic form of the solutions of lovelock gravity to be exactly the same as that of einstein gravity .
thus , we employed the counterterms introduced for einstein gravity in @xcite and found that the power law divergences of static solutions in the action of lovelock gravity can be removed by suitable choice of coefficients .
we found that the counterterms are independent of lovelock coefficients and the dimensionally dependent of them is the same as those of einstein gravity .
the main difference of our work with that of ref .
[ 16 ] is that the counterterms are exactly the same as those of einstein gravity and do not depend on lovelock coefficients .
the only job which remains is that one needs to calculate the coefficients @xmath54 , @xmath55 , @xmath56 and @xmath57 .
for example if one wants to remove the divergences of the action due to the gauss - bonnet term , one should calculate @xmath195 , @xmath196 , @xmath197 and @xmath198 that remove the divergences of @xmath199 , without regarding the other lovelock terms .
this enables one to generalize this method to other higher curvature theories of gravity easily , including fourth order lovelock gravity or @xmath200 gravity .
we also introduced the finite energy - momentum tensor in third order lovelock gravity .
in addition , we employed these counterterms to calculate the finite action and mass of the static black hole solutions of third order lovelock gravity .
calculating the temperature , the electric charge and electric potential and using the calculated finite action and mass , we showed that the entropy calculated through the use of gibbs - duhem relation is consistent with the calculated entropy by wald s formula .
we , also , showed that the conserved and thermodynamic quantities satisfy the first law of thermodynamics .
finally , we investigated the stability of charged black holes of lovelock gravity in canonical ensemble .
we found that the black holes with respect to the sign of @xmath201 given in eq .
( [ eta ] ) may be divided into two classes .
the negative @xmath201 ( small ) black holes show a phase transition between very small and small black holes , while the large black holes are stable .
there is no black hole solution with medium size .
of course by small black holes we mean the size of them with respect to the cosmological parameter @xmath21 . we , also , investigated the effects of third order lovelock term on the stability of the solutions .
we found that there is no large black holes for large negative @xmath108 , while the black holes for small negative @xmath108 are not stable .
we also found that as @xmath108 becomes larger , stable black holes may exist .
this shows that there is a phase transition as the third order term of lovelock gravity becomes larger .
finally , we found that there is no small black hole solution for positive @xmath108 , while the black hole with negative @xmath108 are stable .
finally , we considered the uncharged black holes of lovelock gravity and found that negative @xmath201 solutions are not black holes with event horizon , while the positive @xmath201 black holes show a phase transition .
the first term of eq .
( [ stres ] ) which is the energy - momentum tensor of the surface term is@xmath202 \\ & & + 3\alpha _ { 3}[5p_{ab}-p\gamma _ { ab}+2k\hat{g}_{ab}^{(2)}+\mathcal{l}% _ { 2}(k_{ab}+k\gamma _ { ab})+4j\hat{r}_{ab}-24j_{(a}{}^{c}\hat{r}_{b)c}+8k^{cd}% \hat{r}_{ac}\hat{r}_{bd}-8kk_{a}{}^{c}k_{b}{}^{d}\hat{r}_{cd } \\ & & + 8kk_{ab}k^{cd}\hat{r}_{cd}-8k^{cd}\hat{r}_{ab}\hat{r}_{cd}+16k_{(a}{}^{c}% \hat{r}_{b)}{}^{d}\hat{r}_{cd}+16k_{(a}{}^{c}k_{b)}{}^{d}k_{d}{}^{e}\hat{r}% _ { ce}-8k_{ab}k_{c}{}^{e}k^{cd}\hat{r}_{de}+6j_{ab}\hat{r } \\ & & -8k_{(a}{}^{c}\hat{r}_{b)c}\hat{r}-12j^{cd}\hat{r}_{acbd}-4k^{cd}\hat{r}% \hat{r}_{acbd}-8k_{a}{}^{c}k_{bc}k^{de}\hat{r}% _ { de}+16k_{c}{}^{d}k^{ef}k_{(a}{}^{c}\hat{r}% _ { b)edf}+16k_{d}{}^{f}k^{de}k_{(a}{}^{c}\hat{r}_{b)ecf } \\ & & + 16k_{(a}{}^{c}\hat{r}_{b)dce}\hat{r}^{de}-16kk^{de}k_{(a}{}^{c}\hat{r}% _ { b)dce}-16k^{cd}\hat{r}_{(a}{}^{e}\hat{r}_{b)cde}-4k^{cd}\hat{r}_{ac}{}^{ef}% \hat{r}_{bdef}+8k^{cd}\hat{r}_{c}{}^{e}\hat{r}_{aebd } \\ & & + 8k^{cd}\hat{r}_{c}{}^{e}\hat{r}_{adbe}-8k^{cd}\hat{r}% _ { a}{}^{e}{}_{c}{}^{f}\hat{r}_{bedf}-8k_{(a}{}^{c}\hat{r}_{b)}{}^{def}\hat{r}% _ { cdef}+8k_{a}{}^{c}k_{b}{}^{d}k^{ef}\hat{r}_{cedf}-4k_{ab}k^{cd}k^{ef}\hat{r% } _ { cedf } \\ & & + 8k^{cd}\hat{r}_{a}{}^{e}{}_{b}{}^{f}\hat{r}_{cedf}+2\gamma _ { ab}(\hat{g}% _ { cd}^{(2)}k^{cd}+6j^{cd}\hat{r}_{cd}-j\hat{r}+2kk^{cd}k^{ef}\hat{r}% _ { cedf}-4k_{c}{}^{e}k^{cd}k^{fh}\hat{r}_{dfeh})]\}\end{aligned}\]]and the second term corresponding to the counterterm is @xmath203 \right .   \\ & & \left . -d_{p}\left
[ \frac{2(3n+2)}{4(n-1)}\left [ \left ( \hat{g}_{ab}^{(1)}% \hat{r}^{cd}\hat{r}_{cd}\right ) -d_{a}d_{b}\left ( \hat{r}^{ef}\hat{r}% _ { ef}\right ) + \gamma _ { ab}d^{2}\left ( \hat{r}^{ef}\hat{r}_{ef}\right ) + 2\hat{% r}\hat{r}_{a}^{\;c}\hat{r}_{bc}\right .
\\ & & \left .
+ \gamma _ { ab}d_{c}d_{d}\left ( \hat{r}\hat{r}^{cd}\right ) + d^{2}\left ( \hat{r}\hat{r}_{ab}\right ) -d^{c}d_{b}\left ( \hat{r}\hat{r}% _ { ac}\right ) -d^{c}d_{a}\left ( \hat{r}\hat{r}_{bc}\right ) \right ] -\frac{%
n(n+2)}{16(n-1)^{2}}\left [ -\gamma _ { ab}\hat{r}^{3}\right . \right .   \\ & & \left .
+ 6\left ( \hat{r}^{2}\hat{r}_{ab}-d_{a}d_{b}\hat{r}% ^{2}+\gamma _
{ ab}d^{2}\hat{r}^{2}\right ) \right ] -2\left [ -\gamma _ { ab}\hat{r% } ^{ef}\hat{r}^{cd}\hat{r}_{ecfd}+3\left ( \hat{r}_{a}^{\;e}\hat{r}^{cd}\hat{r}% _ { ecbd}+\hat{r}_{b}^{\;e}\hat{r}^{cd}\hat{r}_{ecad}\right ) \right .
\\ & & \left .
-2d_{c}d_{d}\left ( \hat{r}_{ab}\hat{r}^{cd}\right ) + 2d_{c}d_{d}\left ( \hat{r}_{a}^{\;c}\hat{r}_{b}^{\;d}\right ) + 2\gamma _ { ab}d_{e}d^{f}\left ( \hat{r}^{cd}\hat{r}_{\;cfd}^{e}\right ) + 2d^{2}\left ( \hat{r}^{cd}\hat{r}_{acbd}\right ) \right .
\\ & & \left
. \left . -2d_{e}d_{a}\left
( \hat{r}^{cd}\hat{r}_{\;cbd}^{e}\right ) -2d_{e}d_{b}\left ( \hat{r}^{cd}\hat{r}_{\;cad}^{e}\right ) \right ] + \frac{n-2% } { 2(n-1)}\left [ -d_{a}\hat{r}d_{b}\hat{r}+\hat{r}_{b}{}^{c}d_{c}d_{a}\hat{r}% \right . \right .
\\ & & \left .
+ \hat{r}_{a}{}^{c}d_{c}d_{b}(\hat{r})-2d_{c}d^{c}d_{b}d_{a}(\hat{r}% ) -\frac{7}{2}\hat{r}_{a}{}^{c}{}_{b}{}^{d}d_{c}d{d}(\hat{r})+2d_{a}(\hat{r}% _ { bc})d^{c}(\hat{r})+d_{b}(\hat{r}_{ac})d^{c}(\hat{r})\right .
\\ & & \left .
+ d_{b}\hat{r}_{ac}d^{c}\hat{r}-2d_{c}\hat{r}_{ab}d^{c}\hat{r}-\hat{r% }
_ { b}{}^{c}{}_{c}{}^{d}d_{a}d_{d}\hat{r}-\hat{r}% _ { a}{}^{c}{}_{c}{}^{d}d_{d}d_{b}\hat{r}\right .
\\ & & \left .
-\hat{r}_{a}{}^{c}{}_{b}{}^{d}d_{d}d_{c}(\hat{r})+2\hat{r}% _ { ab}d_{e}d^{e}(\hat{r})+\frac{1}{2}\gamma _ { ab}\left ( d_{e}\hat{r}d^{e}\hat{% r}+d^{4}\hat{r}\right ) \right ] \right .
\\ & & -\left .
\left [ 2d_{a}d_{b}(d^{2}\hat{r})-2d^{4}\hat{r}_{ab}+2d_{p}\hat{r}% _ { ( a}\hat{r}_{b)}^{p}-4\,\hat{r}^{pq}d_{a}d_{b}\hat{r}_{pq}+12\,\hat{r}% ^{pq}d_{(b}\hat{r}_{pa)q}-4(d^{2}\hat{r}^{pq})\hat{r}_{paqb}\right
\\ & & \left .
+ 6d_{p}(\hat{r})d_{(b}\hat{r}_{a)}^{p}-2d_{a}(\hat{r}^{pq})d_{b}(% \hat{r}_{pq})+4d_{(a}\,d_{q}\left ( \hat{r}^{pq}\hat{r}_{b)p}\right ) + 16d^{r}(\,\hat{r}^{pq})d_{(b}\hat{r}_{|rqp|a)}\right .
\\ & & \left .
-4\,\hat{r}^{pq}\hat{r}_{pa}\hat{r}_{qb}+8\,\hat{r}^{pr}\hat{% r}_{r}^{q}\hat{r}_{paqb}+4\,\hat{r}^{pq}\hat{r}_{(a}^{r}\hat{r}_{|rqp|b)}% \right ] \right .
\\ & & + \left .
\gamma _ { ab}\left [ -\,d^{4}\hat{r}-4\,d_{p}d_{q}(\hat{r})% \hat{r}^{pq}+2\hat{r}_{pq}d^{2}(\hat{r}^{pq})+4\,d_{r}d_{s}(\hat{r}_{pq})% \hat{r}^{prqs}-\,d_{e}(\hat{r})d^{e}(\hat{r})\right .
\\ & & \left .
+ 5\,d_{r}(\hat{r}_{pq})d^{r}(\hat{r}^{pq})-8\,d_{r}(\hat{r}% _ { pq})d^{q}(\hat{r}^{pr})-4\,\hat{r}_{pq}\hat{r}_{r}^{p}\hat{r}^{qr}+4\,\hat{% r}_{pq}\hat{r}_{rs}\hat{r}^{prqs}\right ] \right .
\\ & & \left
+ \frac{1}{n-1}\left [ 2\,d_{a}d_{b}(d^{2}\hat{r})-2\,(d^{2}% \hat{r})\hat{r}_{ab}+d_{a}(\hat{r})d_{b}(\hat{r})+\gamma _ { ab}\left ( -2\,d^{4}\hat{r}-\frac{1}{2}\,d_{c}\hat{r}d^{c}\hat{r}\right ) \right ] \right ] \right\ } .\end{aligned}\]]where @xmath54 s , @xmath55 s , @xmath56 s and @xmath57 s are given in eqs .
( a1- [ a3 ] ) .
the above counterterms remove the divergences of the energy - momentum tensor for @xmath204 . as in the case of einstein gravity , one should add more counterterms for @xmath205 .
r. g. cai , z. y. nie and y. w. sun , phys .
d * 78 * , 126007 ( 2008 ) ; r. g. cai , z. y. nie , n. ohta and y. w. sun , phys .
d * 79 * , 066004 ( 2009 ) ; m. blau , k. narain and e. gava , j. high energy phys . * 09 * , 018 ( 1999 ) ; a. buchel , j. t. liu and a. o. starinets , nucl . phys .
b * 707 * , 56 ( 2005 ) ; y. kats and p. petrov , j. high energy phys . *
01 * , 044 ( 2009 ) ; a. buchel , nucl .
b * 803 * , 166 ( 2008 ) ; r. c. myers , m. f. paulos and a. sinha , phys .
d * 79 * , 041901 ( 2009 ) ; a. buchel , r. c. myers , m. f. paulos and a. sinha , phys .
b * 669 * , 364 ( 2008 ) ; d. m. hofman and j. maldacena , j. high energy phys .
* 05 * , 012 ( 2008 ) ; d. m. hofman , nucl .
b * 823 * , 174 ( 2009 ) ; j. de boer , m. kulaxizi and a. parnachev , arxiv:0912.1877 .
m. h. dehghani and m. shamirzaie d * 74 * , 064023 ( 2006 ) ; m. h. dehghani , r. pourhasan , phys . rev .
d * 79 * , 064015 ( 2009 ) ; n. dadhich , s. g. ghosh and s. jhingan , phys .
b * 711 * , 196 ( 2012 ) . ; n. farhangkhah and m. h. dehghani , phys .
d * 90 * , 044014 ( 2014 ) .


in this article we study a type of resampling procedure for markov chains .
the procedure , called weighted ensemble ( we ) sampling  @xcite , simulates some replicas of a markov chain @xmath0 , resampling from the replicas at certain time intervals .
the distinguishing feature of we sampling , compared to other resampling techniques , is that we is designed so that the replicas of @xmath0 are evenly distributed throughout state space .
this is achieved by dividing state space into bins and resampling in each bin so that the number of replicas in a bin remains roughly constant .
the replicas carry probabilistic weights so that the resulting statistical distribution is unbiased .
this distribution can be used , in principle , to estimate any function of of @xmath0 at a fixed time  @xcite .
our interest in we arises from longstanding problems in computational chemistry . in this
setting , @xmath0 is a time discretization of some stochastic molecular dynamics ( md ) .
md simulations have proven useful for understanding many chemical and biological processes ; see  @xcite for an overview . however , such simulations are limited by time scale separation . many phenomena of interest occur at the laboratory time scale of microseconds , while md simulations have time steps that correspond to femtoseconds . in this case ,
straightforward md simulations are not practical .
many methods exist for extending the time scale of md simulations ; we do not attempt to give a review of them here .
however , we mention some related methods , including exact milestoning  @xcite , neus  @xcite , trajectory tilting  @xcite , transition interface sampling  @xcite , forward flux sampling  @xcite , and boxed molecular dynamics  @xcite .
see for instance  @xcite for review and comparison of these methods .
we will compare we to exact milestoning in section  [ sec : dynamics ] below .    in this article
we give a mathematical framework for we based on martingale theory .
the framework allows us to prove that we is exact even when the bins and the number of replicas per bin are chosen adaptively , as noted earlier in  @xcite . while we can be used with a broad range of stochastic processes , when the process is time homogenous and markovian  as in many models of md , such as langevin dynamics  we can be used to efficiently compute dynamical quantities like reaction rates without actually simulating the reactions  @xcite .
these computations rely on hill s relation  @xcite , which we generalize in section  [ sec : dynamics ] below . from hill
s relation , obtaining reaction rates and probabilities requires a calculation on the stationary distribution of a nonreversible process . to speed up this stationary calculation ,
we is combined with a preconditioning step  @xcite in which a markov state model ( msm )  @xcite is used to approximate the stationary distribution .
this is sometimes called accelerated we  @xcite .
accelerated we begins with replicas evenly distributed in space , with weights chosen to match the stationary distribution of the msm .
the replicas are then allowed to relax according to their exact dynamics , with we sampling ensuring that the replicas remain evenly distributed in state space .
we show below that information from the msm can be used to optimize the we sampling in this relaxation step , in the sense that the variance in the appropriate stationary calculation is minimized .
this optimization requires an adaptive number of replicas per bin , in contrast with traditional we sampling .
we show in a simple model that this adaptive sampling can be significantly better than traditional we sampling .    this article is organized as follows . in section  [ sec : assumptions ] , we introduce notation for the replica processes , and we make assumptions on we sampling that we assume hold throughout .
( we refer to the replica processes as `` point and weight '' processes , following familiar notation in the mathematical resampling literature . )
we prove that we is unbiased in a general setting that includes adaptive binning procedures . in section
[ sec : sampling ] , we introduce an explicit algorithm for we sampling ( algorithm  [ alg1 ] ) and prove that it is unbiased ( proposition  [ prop1 ] below ) .
we give an exact formula for the mean squared error of this sampling ( proposition  [ prop2 ] ) .
then , using this error formula , we produce a heuristic near - optimal adaptive strategy . in section  [ sec :
stationary ] , we propose a version of accelerated we which uses this strategy ( algorithm  [ alg2 ] ) . in section
[ sec : example ] , we implement the adaptive strategy on a simple model , and compare with traditional we sampling as well as naive sampling . in section  [ sec : dynamics ] , we show how to obtain certain dynamical averages , including mean first passage times and hitting probabilities , from the stationary distribution of a nonreversible markov chain .
all proofs are in section  [ sec : proofs ] .
in this section we introduce a general mathematical framework for unbiased we sampling . throughout @xmath1
is a markov chain taking values in a measurable space @xmath2 , with transition kernels @xmath3 .
we write @xmath4 for the expectation operator of @xmath1 .
below we consider sampling schemes for @xmath1 of the following type .
let @xmath5 and @xmath6 be a points and weights processes evolving by selection and mutation : @xmath7 here , each @xmath8 and @xmath9 .
we refer to @xmath5 and @xmath6 as the points and weights at time @xmath10 .
the expectation operator associated to the points and weights process will be denoted @xmath11 .    below
we describe rules which guarantee the points and weights process is unbiased .
readers interested only in explicit algorithms can skip to the next section . before describing this process in more detail
, we need some notation . throughout , @xmath12 indicates denote equality in law of random variables or processes .
consider the information generated by the process up to time @xmath10 , that is , the @xmath13-algebra generated by the points and weights up to time @xmath10 , prior to the selection step : @xmath14 consider also the larger @xmath13-algebra obtained when the information from the points and weights in the selection step at time @xmath10 is included : @xmath15    we assume the points and weights process obeys rules ( a1)-(a5 ) below .
the rules do not uniquely define the selection and mutation procedure , but they ensure the points and weights process has certain properties , like being unbiased for the law of @xmath1 ; see theorem  [ thm1 ] below .
we first consider the initialization of the process .
the initial points and weights have the distribution of @xmath16 in the following sense :    * the initial collection of points and weights satisfies @xmath17 w.p . @xmath18 and @xmath19   = { \mathbb e}[h(x_0)].\ ] ] for every bounded measurable @xmath20 .
for example , the initial condition could be an average of a fixed number of samples of @xmath16 , that is , @xmath21 is deterministic and @xmath22 , and @xmath23 for @xmath24 .
notice we do not require that the @xmath25 s are independent , so they can be generated by , for example , markov chain monte carlo or other sequential samplers .
next we consider the selection step . in we
sampling , points are divided according to `` bins '' that partition state space .
we do not require bins here , but we assume that the collection of points @xmath26 at time @xmath10 can be divided into @xmath27 different collections , and that these collections are identifiable at time @xmath10 .
we make the same assumption for the selected points .
* there are partitions @xmath28 of @xmath29 and @xmath30 of @xmath31 that can be determined at time @xmath10 .
that is , @xmath27 and @xmath28 are @xmath32-measurable , and @xmath30 are @xmath33-measurable .
note that the partitions above need not be associated with a partition of state space .
all that is required is a classification of the point indices into disjoint collections .
we assume that the selection is unbiased with respect to this classification :    * the selection in unbiased with respect to the partition in ( a2 ) , that is , @xmath34   = \sum_{i \in
i_p^r }   { \omega}_p^i h_p({\xi_p^i})\ ] ] for every bounded measurable @xmath35 .
we now turn to the mutation step . in this step
, each selected point evolves to form a point of the next generation , with its weight remaining the same .    *
the number of mutated points is the same as the number of selected points : @xmath36 and moreover the weights do not change upon mutation : @xmath37    finally , we assume the mutations are independent and follow the law of @xmath1 .    *
the mutations are obtained by independent draws from @xmath3 : @xmath38    at time @xmath10 , the points and weights process defines an empirical distribution @xmath39 consider the first time all the points have been killed , @xmath40 we adopt the convention that @xmath41 we will now show that @xmath42 is an unbiased approximation for the law of @xmath1 .
this allows us to compute unbiased estimates of functions of @xmath1 at a fixed time @xmath10 .
see the discussion after algorithm  [ alg1 ] below .
[ thm1 ] @xmath43 is an unbiased estimator of the law of @xmath44 .
that is , @xmath45   \equiv { \mathbb e}^{emp}[\eta_n(f_n ) ]   = { \mathbb e}[f_n(x_n)]\ ] ] for every bounded measurable @xmath46 .
we now present a we algorithm that is consistent with the assumptions in section  [ sec : assumptions ] . compared to the general setting above , our algorithm below has two specific features .
first , we use bins to define the partition in ( a2 ) above . and second , the resampling in each bin is designed to produce points in the next generation that all have the same weight ( see also  @xcite ) .
this ensures that the weights do not usually vary by large amounts in small regions of state space .
[ alg1 ] pick initial points @xmath47 and weights @xmath48 as in ( a1 ) .
then iterate :    * _ selection step : _ * choose bins @xmath49 forming a partition of @xmath50 , and a target number of points @xmath51 for each bin @xmath52 .
the @xmath52 s and @xmath51 s must be deterministic functions of the points and weights up to and including the current time @xmath10 .
* for @xmath53 , do the following . set a target weight in bin @xmath52 as @xmath54 for @xmath55 such that @xmath56 , let @xmath57 be a random variable with law @xmath58 let the number of selected points equal to @xmath56 be @xmath57 : @xmath59 assign the target weight @xmath60 to all the selected points in bin @xmath52 : @xmath61 let @xmath62 be the total number of selected points : @xmath63 * _ mutation step : _ * independently evolve the @xmath64 s to obtain the points at time @xmath65 : @xmath66 set the new weights equal to the selected weights : @xmath67 set the total number of new particles as @xmath68 .
then update @xmath69 and return to 1 .
some remarks are in order .
* given the points and weights at time @xmath10 , the procedure in step 2 uniquely defines the law of the selected points and weights , up to index relabeling . from  ,
each point @xmath26 in bin @xmath52 is selected on average @xmath70 times , that is ,
@xmath71 = \frac{\omega_p^i}{{\bar \omega}_p^r},\ ] ] so the expected number of selected points in bin @xmath52 is at time @xmath10 is @xmath72=   \sum_{i:\xi_p^i \in b_p^r } \frac{\omega_p^i}{{\bar \omega}_p^r } = n_p^r.\ ] ] thus , the @xmath51 can be seen as parameters controlling the average number of points per bin . as we have seen in theorem  [ thm1 ] , these numbers may be chosen adaptively . indeed , a key feature of algorithm  [ alg1 ] is that the bins @xmath52 and the numbers @xmath51 may be chosen adaptively . *
the algorithm generates empirical approximations @xmath73 of the distribution of @xmath1 . in proposition  [ prop1 ] below , we show these empirical approximations are unbiased in the sense of theorem  [ thm1 ] . *
recall that @xmath74 is the first time that all the points have been killed .
if @xmath75 is such that @xmath76 $ ] is too large , then the variance of the estimator @xmath77 of @xmath78 $ ] can be large as well .
this is because when @xmath76 $ ] is large , in many simulations the value of @xmath77 will be @xmath79 , which is uninformative . in practice , to avoid this , the values of @xmath51 can be given some lower threshold .
see the remarks after algorithm  [ alg2 ] below . * in practice , the points may not be resampled at every time step .
we have in mind the case where @xmath1 is obtained from an underlying markov chain at certain resampling times .
that is , @xmath80 where @xmath0 is an underlying markov chain and @xmath81 are some resampling times .
the @xmath81 can be deterministic times  for instance @xmath82 , where @xmath83 is a constant lag time  or they can be certain stopping times for @xmath0 . for stationary calculations based on hill s relation , the resampling times should be chosen so that @xmath1 is a time homogeneous markov chain .
see the discussion in section  [ sec : dynamics ] below .    as noted above , algorithm  [ alg1 ] produces unbiased estimates of the law of @xmath44 :    [ prop1 ] algorithm  [ alg1 ] satisfies ( a1)-(a5 ) above when @xmath84 in particular , @xmath43 is an unbiased estimator of the law of @xmath44 in the sense of theorem  [ thm1 ] .    in proposition  [ prop2 ] below we present an explicit formula for the mean squared error in the estimate @xmath85 $ ] when @xmath43 is obtained from algorithm  [ alg1 ] . below proposition
[ prop2 ] , we use the error formula to present an adaptive sampling strategy .
this strategy uses some information from a coarse model ( for instance , a msm ) to guide the sampling .
[ prop2 ] let @xmath75 be a fixed time and @xmath46 a bounded measurable function . in algorithm  [ alg1 ] , the mean squared error in the approximation @xmath77 of @xmath78 $ ] satisfies @xmath86\right)^2\right ]   = { \mathbb e}^{emp}[(\eta_0(f_0)-{\mathbb e}[f_n(x_n)])^2]\\ & \qquad \qquad \qquad\qquad \qquad\qquad
\qquad + \sum_{p=1}^n   { \mathbb e}^{emp}\left[\left ( \eta_p(f_p)-\eta_{p-1}(f_{p-1})\right)^2 \right ] \end{split}\end{aligned}\ ] ] where , writing @xmath87 $ ] and @xmath88 , we have @xmath89\\ & \qquad   =   \sum_{r=1}^{r_p } \sum_{i:\xi_p^i \in b_p^r}\left [ { \bar \omega}_p^r\omega_p^i k_p g_{p+1}^2(\xi_p^i)- \left({\bar \omega}_p^r\omega_p^i + \left(\omega_p^i\right)^2 -\left({\bar \omega}_p^r\right)^2\sigma_p^{r , i}\right ) g_p(\xi_p^i)^2\right ] . \end{split}\end{aligned}\ ] ]    given the points and weights at time @xmath10 , we may try to minimize the contributions to the mean squared error in   over choices of @xmath52 and @xmath51 .
we pursue this idea , for fixed bins but an adaptive target number of points per bin .
suppose for given bins @xmath52 we want to optimize over the allocation @xmath51 of points , subject to the constraint that the average total number of points is fixed at @xmath90 .    for the remainder of this section
, we assume that @xmath1 is time homogeneous with transition kernel @xmath91 , that the bins @xmath92 do not change in time , with @xmath93 total bins , and that @xmath94 . using the mean squared error formula in proposition  [ prop2 ] , we heuristically derive an adaptive sampling strategy below .
notice that @xmath57 are defined so that they have minimal variance .
we estimate @xmath95 \approx   { \mathbb e}^{emp}\left[\left .
c_p^{r , i}\right| { \mathcal f}_p\right]^2   = \left(\frac{\omega_p^i}{{\bar \omega}_p^r}\right)^2.\ ] ] suppose we have approximations @xmath96 of @xmath97 inside @xmath98 , that is , @xmath99 in this case ,   rewrites as @xmath100\\ & = \sum_{r=1}^{r_p } \sum_{i:\xi_p^i \in b_p^r}\left [ { \bar \omega}_p^r\omega_p^i k g_{p+1}^2(\xi_p^i)-
\left({\bar \omega}_p^r\omega_p^i + \left(\omega_p^i\right)^2 -\left({\bar \omega}_p^r\right)^2\sigma_p^{r , i}\right ) g_p(\xi_p^i)^2\right ] \\ & \approx \sum_{r=1}^{r_p } \sum_{i:\xi_p^i \in b_p^r } { \bar \omega}_p^r\left[\omega_p^i k g_{p+1}^2(\xi_p^i)-   \omega_p^i g_p(\xi_p^i)^2\right ] \\ & \approx \sum_{r=1}^{r_p } { \bar \omega}_p^r v_p^r \sum_{i:\xi_p^i \in b_p^r } \omega_p^i \\ & = \sum_{r=1}^r \frac{v_p^r \left(\sum_{i:\xi_p^i \in b_p^r } \omega_p^i\right)^2}{n_p^r}. \end{split}\end{aligned}\ ] ] minimizing the last expression with the constraint @xmath101 leads to the formula @xmath102 the approximations @xmath96 can be obtained , for instance , by estimating a coarse probability @xmath103 of @xmath1 going from bin @xmath98 to bin @xmath104 in one time step .
this essentially amounts to constructing a msm  @xcite for @xmath1 , with each state corresponding to a bin , and with time step @xmath18 .
of course , as with any msm , there is error arising from the markov assumption , since the probability @xmath103 for @xmath1 to transition between bins @xmath98 and @xmath104 depends on the initial distribution of @xmath1 in @xmath98 .
however , the msm can be seen as a coarse model for @xmath1 , and in particular , when @xmath1 has stationary distribution @xmath105 , we will use the stationary distribution of the msm as an initial guess of @xmath105 .
some comments on the term @xmath106 are in order .
it is not hard to see that @xmath107.\ ] ] the heuristic strategy suggests the larger the value of @xmath108 , the more beneficial it is to select @xmath109 .
it is also easy to check that for naive sampling  that is , sampling with no selection step  the contributions to the mean squared error are @xmath110    = \sum_{i=1}^n \left[k g_{p+1}^2(\xi_p^i )    - g_p^2(\xi_p^i)\right],\ ] ] with @xmath90 the ( constant ) total number of points .
we explore the heuristic strategy above in section  [ sec : stationary ] below , showing how it can be combined with existing we technologies to efficiently sample the stationary distribution of @xmath1 in the time homogeneous case .
in this section we present a procedure , algorithm  [ alg2 ] below , for computing stationary averages in the time homogeneous case . in this section , @xmath1 is a time homogeneous markov chain with values in a measurable space @xmath111 , and transition kernel @xmath112 .
we assume @xmath1 has a unique stationary distribution  @xmath105 .
the algorithm below is designed to efficiently compute @xmath113 for a fixed bounded measurable function @xmath114 .
[ alg2 ] choose a relaxation time @xmath75 , a sampling measure @xmath109 on @xmath111 , and bins @xmath115 forming a partition of @xmath116 .
choose a target number of total points @xmath117 .
then iterate : _ preconditioning step : _    *
let @xmath118 estimate the probability for @xmath1 to go from @xmath98 to @xmath104 in one step : @xmath119\,\xi(dx)\ ] ] let @xmath120 estimate the value of @xmath121 inside bin @xmath98 : @xmath122 * let @xmath123 be the stationary distribution for the transition matrix @xmath124 .
that is , @xmath125 is the left eigenvector of @xmath126 corresponding to the eigenvalue @xmath18 , normalized so that @xmath127 .
* let @xmath96 be the @xmath128th entry of the vector @xmath129 , where @xmath130 is the vector estimate of @xmath121 and the squaring operations are entrywise .
_ relaxation step : _
* choose initial points so that the number of points in @xmath98 is @xmath131 : @xmath132 choose initial weights so that the total weight inside each bin @xmath98 is @xmath133 : @xmath134 * proceed through algorithm  [ alg1 ] from time @xmath135 to @xmath136 , with either @xmath137 or @xmath138 defining the target number of points in bin @xmath98 at time @xmath10 . *
the output of the algorithm is @xmath139 .
some remarks are in order .    *
the purpose of the preconditioning step is to obtain a coarse approximation @xmath123 of @xmath105 , the stationary distribution of @xmath1 .
this approximation is then used as the initial condition of the relaxation step .
note that @xmath103 and @xmath120 can be estimated by running many short trajectories in parallel .
the sampling measure @xmath109 is the initial distribution for these short trajectories .
* @xmath103 is essentially the transition matrix of a msm for @xmath1 defined so that its states are the bins @xmath140 , and its time step is exactly @xmath18 .
if @xmath1 is obtained from an underlying markov chain @xmath0 via @xmath141 , then this can be viewed as an msm for @xmath0 with time step @xmath83 . under suitable conditions ,
the vector @xmath123 is the unique stationary distribution of the msm . to understand the expression for @xmath96 , note that since @xmath142 , @xmath143 thus , @xmath96 is defined using  , where the approximation is obtained by replacing @xmath112 with @xmath126 and @xmath121 with @xmath144 on the right hand side of that equation . in particular ,
the definition of @xmath96 is consistent with the heuristic calculations in section  [ sec : sampling ] . *
the purpose of the relaxation step is to allow the coarse approximation @xmath123 to relax to the stationary distribution @xmath105 .
when the relaxation time @xmath75 is appropriately chosen , @xmath145 is a good approximation of @xmath113 .
since the relaxation step starts at the coarse approximation @xmath123 of @xmath105 , the relaxation time @xmath75 can be much smaller than the typical time to converge from an arbitrary initial distribution .
* when @xmath146 as in  , we recover the accelerated weighted ensemble technique of  @xcite . on the other hand , when @xmath147 satisfies   the sampling is adaptive . for comparisons of sampling with   and  ,
see the simulations in section  [ sec : example ] below . in the adaptive case ,
the choice of @xmath51 depends on the parameter @xmath75 .
thus , a simulation at a given value of @xmath75 can not be easily extended to a larger value of @xmath75 with the same formula   defining @xmath51 .
also , in the adaptive case the choice of @xmath51 depends on @xmath121 , and so separate simulations are likely needed for computing @xmath148 $ ] for different functions @xmath121 . *
the @xmath149 is introduced so that @xmath150 with high probability .
when @xmath149 is too small , it can happen that all the points are killed in a large fraction of simulations .
in this situation , the variance can be large .
this killing can occur if , for example , @xmath51 is nonzero only in regions where there are no points . on the other hand ,
if @xmath149 is too large , we can lose the benefits of the adaptive sampling .
here we consider an underlying markov chain @xmath0 on state space @xmath151 with transition matrix @xmath152 where @xmath153 and @xmath154 is chosen so that @xmath155 is stochastic .
we take resampling intervals @xmath156 , so @xmath157 and the transition matrix of @xmath1 is @xmath158 .
the bins will be @xmath159 thus , there are @xmath160 fixed bins .
let @xmath105 be the stationary distribution of @xmath1 , and @xmath161 we also let @xmath162 be its normalized version .
we use algorithm  [ alg2 ] to obtain empirical approximations @xmath145 of @xmath113 for a fixed relaxation time  @xmath75 . throughout
, we use a sampling measure @xmath109 which is uniform : @xmath163 for all @xmath55 . in the relaxation step of algorithm  [ alg2 ] ,
the initial empirical distribution of points and weights can be considered a sample from @xmath164    we will use three types of sampling .
the first type of sampling uses the adaptive allocation of points from equation   above .
we call this adaptive we sampling . in  , we take @xmath165 and @xmath166 .
we found this @xmath149 to be sufficiently large that @xmath150 in all our simulations .    in the second type of sampling we used a fixed target number of points per bin .
we call this traditional we sampling .
it is the sampling method described in  @xcite .
here we use equation  , taking @xmath167 .
again , we had @xmath150 in all our simulations .
notice for both adaptive and traditional we sampling , we have chosen @xmath90 so that the target number of points is @xmath168 .    [ fig1]-190pt     vs. @xmath75 from the example in section  [ sec : example ] from adaptive , traditional and naive sampling .
the crosses are exact values corresponding to @xmath169 , and the dotted line is the stationary value @xmath113 .
bottom right : empirical standard deviations @xmath170 of @xmath145 , obtained from @xmath171 independent simulations .
( these are the error bar radii from the other plots . ) ]    -170pt    [ fig2]-195pt     at time @xmath172 , compared to @xmath173 , the normalized version of @xmath121 . here
, @xmath174 is the total number of points at time @xmath75 .
bottom right : stationary distribution @xmath105 .
]    -170pt    the last type of sampling does not use resampling at all .
we call this naive sampling . here , we skip steps ( ii ) and ( iii ) of the relaxation step of algorithm  [ alg2 ] . instead , after the points @xmath175 and weights @xmath176 are chosen , we simply evolve these points independently until time @xmath75 , without changing the weights . for this sampling
we take @xmath165 total points , to facilitate comparison with the traditional and adaptive sampling described above .
results comparing the different relaxation techniques are in figures  [ fig1]-  [ fig2 ] . in figure
[ fig1 ] , we plot @xmath145 vs. @xmath75 for various values of @xmath75 , showing convergence to the stationary value @xmath113 .
we compute error bars as empirical standard deviations from @xmath171 independent simulations .
we found that the error bars for adaptive we sampling were significantly smaller than that of traditional we and naive sampling . in figure
[ fig2 ] , we plot histograms representing the average distribution of the points @xmath177 at time @xmath75 .
note that traditional we sampling distributes the points roughly uniformly in space , as expected , while adaptive we sampling guides the points towards the region in state space relevant for computing @xmath121 .
meanwhile , naive sampling distributes the points according to the stationary distribution @xmath105 . in figure
[ fig3 ] , we plot the estimates @xmath96 from the adaptive sampling strategy for @xmath178 and @xmath179 where @xmath180 is the relaxation time
. note that by time @xmath181 , the sampling is focused near the support of @xmath121 .
when @xmath121 is a function with large values in regions of low @xmath105 probability , as in this example , naive sampling performs poorly compared to both traditional and adaptive we sampling .
when state space is very large compared to the region @xmath182 where @xmath121 has large values ( or is non - negligible ) , we expect adaptive we sampling to perform much better than traditional we sampling , due to the fact that traditional we sampling will distribute the points very thinly throughout space , including in @xmath182 , while adaptive we sampling will push most of the points towards @xmath182 .
a possible drawback of adaptive we sampling is that it requires more computations at the resampling times , compared to traditional we sampling .
however , in practice the resampling times may be large enough so that this extra effort contributes little to the overall computational cost .
finally , we note that the adaptive sampling above can also be used more generally to estimate time marginals of @xmath44 , that is , expectations of the form @xmath148 $ ] at fixed finite times @xmath75 , from an arbitrary initial distribution of @xmath16 . in this case , a msm is still required to guide the sampling .
one of the advantages of the adaptive sampling in the stationary case is that an msm has already been computed as part of a preconditioning step .
[ fig3]-210pt     from   vs. @xmath10 for the example in section  [ sec : example ] . here
, we take @xmath135 and @xmath183 , when @xmath180 is the final time . ]
in this section we show how to compute certain dynamical averages of time homogeneous markov chains from stationary calculations on a related markov chain .
the hill relation  @xcite shows that a mean first passage time can be reformulated as a certain stationary average .
we will generalize this relation below .
assume that @xmath1 is obtained from an underlying markov chain @xmath0 at some resampling times @xmath81 , that is , @xmath80 .
we assume @xmath1 and @xmath0 have values in a measurable space @xmath111 .
@xmath0 can be continuous or discrete in time . for simplicity , and since in simulations time discretization is required anyway , we assume here it is discrete in time . to obtain @xmath1 from @xmath0 , we need to define the resampling times @xmath81 .
we will assume they satisfy the following :    * @xmath184 are stopping times for @xmath0 such that for all @xmath185 , @xmath186    informally , this means @xmath0 and the stopping time increments @xmath187 regenerate at each stopping time @xmath81 .
for example , we can take @xmath81 to be @xmath10 multiplied by some deterministic lag time .
alternatively , the @xmath81 can be a sequence of hitting times of coarse sets in state space , as in exact milestoning ; see the comments below .    to be able to compute dynamical averages from stationary ones
, we must introduce a source and sink .
let @xmath188 and @xmath189 be disjoint initial and final sets , such that when @xmath0 arrives in @xmath189 at one of the resampling times , it immediately returns to @xmath188 .
when it returns to @xmath188 , it starts at a fixed source distribution @xmath190 on @xmath188 :    * there are disjoint measurable sets @xmath191 and a distribution @xmath190 on @xmath188 so that @xmath192 =   \int { \mathbb p}^x[y_1 \in a]\rho(dx).\ ] ]    we are interested in computing , for example , the mean first passage time of @xmath0 from @xmath188 to @xmath189 .
we want to compute this from a stationary average , using the source - sink .
the following technical assumption guarantees this computation is well defined :    * with @xmath193 we have @xmath194 < \infty , \qquad \text{for all } x \in e.\ ] ]    we assume ( b1)-(b3 ) hold throughout the remainder of this section . from ( b1 ) , it is easy to see @xmath195 is a stopping time for @xmath0 . since @xmath80 ,
by the strong markov property , ( b1 ) guarantees @xmath1 is a time homogeneous markov chain . from ( b2)-(b3 )
, @xmath1 has a unique stationary distribution @xmath105 .
the latter can be proved by standard coupling arguments , based on @xmath1 reaching @xmath189 in finite expected time and then regenerating at @xmath190 .
we omit proof .    for the resampling times @xmath81 , we could take @xmath82 where @xmath83 is a fixed resampling time interval , as in standard we sampling .
more generally , we could take @xmath196 and @xmath197 where @xmath198 is a stopping time for @xmath0 starting at @xmath199 , as in exact milestoning . in that setting , @xmath198 is the first time for @xmath0 to reach a milestone starting at @xmath199 ; see  @xcite .
we are now ready to generalize the hill relation  @xcite .
[ thm_dynamic ] with @xmath105 the unique stationary distribution of @xmath1 , @xmath200 = \frac{\int_f g(y)\pi(dy)}{\pi(f)}\ ] ] and @xmath201 =   \frac{{\mathbb e}^\pi\left[\sum_{t=1}^{\tau_1 } g(y_t)\right ]   } { \pi(f)}.\ ] ]    in  , if @xmath202 is the disjoint union of @xmath203 and @xmath204 and @xmath205 , we have @xmath206 =   \frac{\pi(b)}{\pi(a\cup b)}.\ ] ] this is the probability to hit @xmath204 before @xmath203 , starting at @xmath190 . in  , if @xmath207 , @xmath208 = \frac{{\mathbb e}^\pi[\tau_1]}{\pi(f)}.\ ] ] this is the mfpt from @xmath190 to @xmath189 . note that when @xmath209 is constant , this equation is simply the hill relation  @xcite .
when @xmath210 is random , this is the expression recently proved for exact milestoning  @xcite .
it is important to note that assumption ( b2 ) does not affect the averages on the left hand side of   and  .
thus , @xmath0 can be any time homogeneous markov chain , and in particular it need not have a source and sink .
the source and sink in ( b2 ) are only needed to define the stationary distribution @xmath105 used in the sampling .    since here @xmath105 is the stationary distribution of a time homogeneous markov chain , algorithm  [ alg2 ]
can be used to compute the right hand side of  .
see the example in section  [ sec : stationary ] above .
although it is not obvious how to extend that algorithm to compute the right hand side of  , in some cases a simple modification may suffice .
for instance , in exact milestoning , one can apply algorithm  [ alg2 ] to the function @xmath211 , where @xmath212 is the union of the milestones .
the empirical distributions @xmath43 from the algorithm can be used as the initial distribution for computing the numerator of  .
we begin with the following preliminary result .    [ lem1 ]
suppose @xmath213 are iid random variables and @xmath90 is a nonnegative integer valued random variable independent of the @xmath214 s .
then @xmath215   = { \mathbb e}[n]{\mathbb e}[\xi_1]\ ] ] and @xmath216 = \left({\mathbb e}[n^2]-{\mathbb e}[n]\right){\mathbb e}[\xi_1]^2 + { \mathbb e}[n]{\mathbb e}[\xi_1 ^ 2].\ ] ]    we omit the proof , which is straightforward .
we will also use the following result .
[ lem2 ] let @xmath217 where @xmath218 .
then we have @xmath219 = g_p({\hat \xi}_p^i)\ ] ] and @xmath220 =    \begin{cases } g_p({\hat
\xi}_p^i ) g_p({\hat \xi}_p^j ) , & i\ne j\\ k_p g_{p+1}^2({\hat \xi}_p^i ) ,   & i = j\end{cases}.\ ] ]    first observe that , by ( a5 ) , @xmath221 \\ & = { \mathbb e}^{emp}\left[\left .
\int k_{p+1}(\xi_{p+1}^i , dy_{p+2})\ldots   k_{n-1}(y_{n-1},dy_{n})f(y_{n})\right|\hat{{\mathcal f}_p}\right ] \\ &
= \int k_p({\hat \xi}_p^i , dy ) k_{p+1}(y , dy_{p+2})\ldots   k_{n-1}(y_{n-1},dy_{n})f(y_{n})= g_p({\hat \xi}_p^i).\end{split}\end{aligned}\ ] ] when @xmath222 , in the second equation , using ( a5 ) we have @xmath223   = \int k_p({\hat \xi}_p^i , dy )   g_{p+1}(y)^2 = k_p g_{p+1}^2({\hat \xi}_p^i).\ ] ] on the other hand , when @xmath224 , conditionally on @xmath225 , @xmath226 and @xmath227 are independent .
thus , appealing to   completes the proof .
we now turn to the proof of theorem  [ thm1 ] .
let @xmath228 be as in lemma  [ lem2 ] and recall that @xmath229 to emphasize that @xmath230 when @xmath231 , we write @xmath232 let @xmath233 note that @xmath234 is @xmath235-measurable and @xmath236 using also @xmath237 we find that @xmath238   & = { \mathbb e}^{emp}\left[\left.\mathbbm{1}_{\tau_{kill } > p+1}\sum_{i=1}^{n_{p+1 } } \omega_{p+1}^i g_{p+1}(\xi_{p+1}^i)\right|{\mathcal f}_p\right ] \\ & = \mathbbm{1}_{\tau_{kill } > p}{\mathbb e}^{emp}\left[\left.{\mathbb e}^{emp}\left[\left.\sum_{i=1}^{n_{p+1 } } \omega_{p+1}^i g_{p+1}(\xi_{p+1}^i)\right|{\hat{\mathcal f}}_p\right]\right|{\mathcal f}_p\right ] \\ & = \mathbbm{1}_{\tau_{kill } > p } { \mathbb e}^{emp}\left[\left.\sum_{i=1}^{{\hat n}_{p } } { \hat \omega}_{p}^i   { \mathbb e}^{emp}\left[\left .
g_{p+1}(\xi_{p+1}^i)\right|{\hat{\mathcal f}}_p\right]\right|{\mathcal f}_p\right ] \qquad \textup { by ( a4 ) } \\ & = \mathbbm{1}_{\tau_{kill } >
p}{\mathbb e}^{emp}\left[\left.\sum_{i=1}^{{\hat n}_p } { \hat \omega}_p^i   g_p({\hat \xi}_p^i)\right|{\mathcal f}_p\right ] \qquad \textup { by ( a5 ) and lemma~8}\\ & = \mathbbm{1}_{\tau_{kill } > p}\sum_{r=1}^{r_p}{\mathbb e}^{emp}\left[\left.\sum_{i\in { \hat i}_p^r } { \hat \omega}_p^i   g_p({\hat \xi}_p^i)\right|{\mathcal f}_p\right ] \qquad \text{by ( a2 ) }   \\ & = \mathbbm{1}_{\tau_{kill } > p}\sum_{r=1}^{r_p}\sum_{i\in { i}_p^r } { \omega}_p^i   g_p({\xi}_p^i ) \qquad \text{by ( a3)}\\ & = \mathbbm{1}_{\tau_{kill } > p}\sum_{i=1}^{n_p } { \omega}_p^i   g_p({\xi}_p^i ) = m_p \qquad \text{by ( a2)}.\end{aligned}\ ] ] it follows that @xmath239 is a martingale , so @xmath240   = { \mathbb e}^{emp}[m_n]$ ] and @xmath241   & = { \mathbb e}\left[g_0(x_0)\right ] \qquad \\ & = { \mathbb e}^{emp}[m_0 ]   \qquad \textup{by ( a1)}\\ & = { \mathbb e}^{emp}[m_n ]   = { \mathbb e}^{emp}[\eta_n(f_n ) { \mathbbm{1}}_{\tau_{kill}>n}].\end{aligned}\ ] ]    clearly , algorithm  [ alg1 ] satisfies ( a1)-(a2 ) and ( a4)-(a5 ) .
conditional on @xmath32 , for @xmath242 such that @xmath243 , the @xmath244 s are independent with expected value @xmath245 .
conditional on @xmath32 , the expected number of such @xmath242 s is @xmath246 = { \mathbb e}^{emp}\left[\left .
c_p^{r , i}\right|{\mathcal f}_p\right ] = \frac{\omega_p^i}{{\bar \omega}_p^r}.\ ] ] thus , for any bounded measurable @xmath247 , @xmath248   & = { \bar \omega}_p^r{\mathbb e}^{emp}\left[\left.\sum_{i:{\hat \xi}_p^i \in b_p^r } h_p({{\hat \xi}_p^i } ) \right| { \mathcal f}_p\right ]   \\   & = { \bar \omega}_p^r \sum_{i : \xi_p^i \in b_p^r}{\mathbb e}^{emp}\left[\left .   \sum_{j:{\hat
\xi}_p^j = \xi_p^i } h_p({{\hat \xi}_p^j } ) \right| { \mathcal f}_p\right ]   \\    & = { \bar \omega}_p^r \sum_{i : \xi_p^i \in b_p^r } \frac{\omega_p^i}{{\bar \omega}_p^r}h_p ( { \xi}_p^i ) \qquad \text{by lemma } 7   \\ & = \sum_{i \in i_p^r }   { \omega}_p^i h_p({\xi_p^i}).\end{aligned}\ ] ]    define @xmath249 and @xmath250 as in the proof of theorem  [ thm1 ] .
there it was shown @xmath239 is a martingale , which leads to equation  [ l2error ] .
we turn to the expression in  . using @xmath251
we find that @xmath252 \\ & = { \mathbb e}^{emp}\left[\left.\mathbbm{1}_{\tau_{kill}>p+1}\left(\sum_{i=1}^{n_{p+1 } } \omega_{p+1}^i g_{p+1}(\xi_{p+1}^i)\right)^2\right|{\mathcal f}_{p}\right ] \\ & = \mathbbm{1}_{\tau_{kill}>p}{\mathbb e}^{emp}\left[\left.\sum_{i=1}^{n_{p+1 } } \sum_{j=1}^{n_{p+1 } } \omega_{p+1}^i   \omega_{p+1}^j g_{p+1}(\xi_{p+1}^i ) g_{p+1}(\xi_{p+1}^j)\right|{\mathcal f}_{p}\right ] \\ & = \mathbbm{1}_{\tau_{kill}>p}\sum_{r=1}^{r_{p } } \sum_{s=1}^{r_{p } } { \bar \omega}_{p}^r { \bar \omega}_{p}^s    { \mathbb e}^{emp}\left[\left.\sum_{i:{\hat \xi}_{p}^i \in b_p^r } \ , \sum_{j:{\hat \xi}_{p}^j \in b_p^s }   g_{p+1}(\xi_{p+1}^i ) g_{p+1}(\xi_{p+1}^j)\right|{{\mathcal f}}_{p}\right]\\ & = \mathbbm{1}_{\tau_{kill}>p}\sum_{r=1}^{r_{p } } \sum_{s=1}^{r_{p}}{\bar \omega}_{p}^r { \bar \omega}_{p}^s \sum_{i:{\xi}_{p}^i \in b_p^r } \ , \sum_{j:{\xi}_{p}^j \in b_p^s } { \mathbb e}^{emp}\left[\left . \sum_{k : { \hat \xi}_{p}^k = \xi_{p}^i }   \sum_{\ell:{\hat \xi}_{p}^\ell = \xi_{p}^j }    g_{p+1}(\xi_{p+1}^k ) g_{p+1}(\xi_{p+1}^\ell)\right|{{\mathcal f}}_{p}\right].\end{aligned}\ ] ] from lemma  [ lem2 ] , @xmath253 = \begin{dcases } g_p({\hat
\xi}_p^k)g_p({\hat \xi}_p^\ell ) , & k\ne \ell \\
k_p g_{p+1}^2({\hat \xi}_p^k ) , & k = \ell \end{dcases}\end{aligned}\ ] ] and recall that @xmath254= { \mathbb e}^{emp}\left[\left.\#\left\{k:{\hat \xi}_p^k = \xi_p^i \in b_p^r\right\}\right|{\mathcal f}_p\right ] = \frac{\omega_p^i}{{\bar \omega}_p^r}.\ ] ] thus , for @xmath255 such that @xmath256 and @xmath257 , by lemma  [ lem1 ] , @xmath258 =   \frac{\omega_{p}^i}{{\bar \omega}_{p}^r } \frac{\omega_{p}^j}{{\bar \omega}_{p}^s }   g_p(\xi_p^i)g_p(\xi_p^j).\end{aligned}\ ] ] on the other hand , when @xmath222 and @xmath259 , this becomes @xmath260 \\   & \qquad \qquad = \left(\sigma_p^{r , i}-\frac{\omega_p^i}{{\bar \omega}_p^r}\right ) g_p(\xi_p^i)^2 + \frac{\omega_p^i}{{\bar \omega}_p^r } k_p g_{p+1}^2(\xi_p^i).\end{aligned}\ ] ] where again we used lemma  [ lem1 ] , and we recall that @xmath261.\ ] ] combining the above , @xmath252 = \mathbbm{1}_{\tau_{kill}>p}\sum_{r=1}^{r_p } \sum_{s=1}^{r_p}\sum_{\substack{i:\xi_p^i \in b_p^r\\ j:\xi_p^j \in b_p^s\\ i \ne j } } \omega_p^i \omega_p^j g_p(\xi_p^i)g_p(\xi_p^j ) \\ & \qquad \qquad+\mathbbm{1}_{\tau_{kill}>p}\sum_{r=1}^{r_p } \sum_{i:\xi_p^i \in b_p^r}\left [   \left(\left({\bar \omega}_p^r\right)^2\sigma_p^{r , i}-{\bar \omega}_p^r\omega_p^i\right ) g_p(\xi_p^i)^2 + { \bar \omega}_p^r\omega_p^i k_p g_{p+1}^2(\xi_p^i)\right],\end{aligned}\ ] ] and so , since @xmath262 we get @xmath263 \\ & =   { \mathbb e}^{emp}\left[\left .
m_{p+1}^2 \right|{\mathcal f}_p\right]-m_p^2 \\
& = \mathbbm{1}_{\tau_{kill}>p}\sum_{r=1}^{r_p } \sum_{i:\xi_p^i \in b_p^r}\left [ { \bar \omega}_p^r\omega_p^i k_p g_{p+1}^2(\xi_p^i)- \left({\bar \omega}_p^r\omega_p^i + \left(\omega_p^i\right)^2 -\left({\bar \omega}_p^r\right)^2\sigma_p^{r , i}\right ) g_p(\xi_p^i)^2\right].\end{aligned}\ ] ]    we begin with the proof of  .
recall @xmath264 is the transition kernel of @xmath265 .
conditioning on @xmath266 and using assumption ( b1 ) , the lhs of   rewrites @xmath267 & =   \int { \mathbb e}^x\left[\left.g(y_{\tau_f})\right|y_{\tau_1 } = y\right]k(x , dy ) \\ &
= \int_{e \setminus f } { \mathbb e}^x\left[\left.g(y_{\tau_f})\right|y_{\tau_1 } = y\right]k(x , dy ) + \int_f g(y)k(x , dy)\\ & = \int_{e \setminus f }   { \mathbb e}^y[g(y_{\tau_f})]k(x , dy )   + \int_f g(y)k(x , dy).\end{aligned}\ ] ] integrating against @xmath105 and using stationarity ,
@xmath268 = \int_{e \setminus f } { \mathbb e}^y[g(y_{\tau_f})]\pi(dy ) + \int_f   g(y)\pi(dy).\ ] ] cancelling terms and using the fact that @xmath269 , @xmath270\pi(dy ) =   \pi(f ) { \mathbb e}^\rho[g(y_{\tau_f})].\ ] ] now we turn to the proof of  .
conditioning on @xmath266 and using assumption ( b1 ) , @xmath271   & = \int { \mathbb e}^x\left[\left.\sum_{t=1}^{\tau_f } g(y_t)\right|y_{\tau_1 } = y\right]k(x , dy ) \\ & = \int { \mathbb e}^x\left[\left.\sum_{t=1}^{\tau_1 } g(y_t)\right|y_{\tau_1 } = y\right]k(x , dy ) \\ &
\qquad\qquad+\int_{e \setminus f}{\mathbb e}^x\left[\left.\sum_{t=\tau_1 + 1}^{\tau_f } g(y_t)\right|y_{\tau_1 } = y\right]k(x , dy)\\ & = { \mathbb e}^x\left[\sum_{t=1}^{\tau_1 } g(y_t)\right ] + \int_{e \setminus f}{\mathbb e}^y\left[\sum_{t=1}^{\tau_f } g(y_t)\right]k(x , dy).\end{aligned}\ ] ] integrating against @xmath105 and using stationarity , @xmath272   = { \mathbb e}^\pi\left[\sum_{t=1}^{\tau_1 } g(y_t)\right ]   + \int_{e \setminus f}{\mathbb e}^y\left[\sum_{t=1}^{\tau_f } g(y_t)\right ] \pi(dy).\ ] ] cancelling terms and using the fact that @xmath273 , @xmath274 = \int_f { \mathbb e}^y\left[\sum_{t=1}^{\tau_f}g(y_t)\right ] \pi(dy ) = \pi(f){\mathbb e}^\rho\left[\sum_{t=1}^{\tau_f}g(y_t)\right].\ ] ]
d. aristoff would like to acknowledge enlightening conversations with tony lelivre , petr plech , mathias rousset , gideon simpson , ting wang , and dan zuckerman .
d. aristoff also gratefully acknowledges support from the national science foundation via the award nsf - dms-1522398 .