aerosol particles have a direct and indirect impact on the earth s climate .
one of the most important physical properties of aerosol particles is their size , and the concentration of particles in terms of their size is referred to as the particle size distribution .
an important characteristic of these data is that because aerosol particles are governed by formation and transformation processes they tend to form well distinguished modal features . investigating these features
provides an understanding of the dynamic behaviour of aerosol particles , their effect on the climate and their association with adverse health effects .
this type of data is increasingly being measured on a regular basis , with the potential to provide more detailed information than , for example , measurements of particle mass concentrations such as pm10 or pm2.5 , traditionally used for regulatory purposes [ @xcite ] .
the most common approach for representing particle size distributions is by treating the size distribution at any time point as a set of individual typically normal distributions or modes [ @xcite , @xcite ] . in this formulation
the estimation of particle size distributions is then analagous to a finite parametric mixture model problem at each time point .    while interest is in the representation of the particle size distribution as a mixture at each time point , it is also of interest to describe how this distribution evolves over time . to better understand aerosol dynamic processes ,
a feature of the measurements of particle size distributions is that they are often collected at regular points in time , and often at quite small time intervals ( e.g. , every 10 minutes ) . in this
setting , parameters of the mixture model at each time point are likely to be correlated with neighboring time points and useful information about the parameters may be gained by incorporating this information in estimation .
the standard setting in which mixture models have been applied has largely been for independent random samples [ @xcite ] , but literature is developing for situations in which the data are spatially and/or temporally structured [ @xcite , @xcite , @xcite , @xcite , @xcite , @xcite ] . the development has largely been driven by the increasing availability of information in a wide variety of applications .
an example includes analysing images ( cat scan ) of sheep over time in which interest is in changes to the composition of fat , bone and tissue [ @xcite ] . in @xcite interest
is in cell fluorescent imaging tracking modelled using a dynamic spatial point process and a mixture representation for the different intensity functions observed .
for the air pollution example considered in this paper , we are interested in a  mixture representation using a missing data approach , in which the components themselves can be interpreted as potential substrata of the data and for which further interest is in their behaviour over time . in particular , we are interested in the evolution of the parameters of the components over time , which in the case of the particle size distribution data is able to reveal important information about the number and change in size of particles for particular modes along with a measure of their variation .
this information can then be used to better understand the potential variables affecting the dynamic behaviour of each mode ( e.g. , from local effects such as combustion from petrol and diesel vehicle engines , construction activity , wind speed , temperature , etc . , and regional effects ) , which are likely to vary substantially between modes , and provide for a more accurate risk assessment of potential effects on adverse health outcomes ( e.g. , respiratory illnesses ) .
popular recent approaches that allow for the correlated nature of the parameters in a mixture setting , both within and across epochs , include dependent dirichlet process mixture models ( ddpm ) and ( spatial ) dynamic factor models ( sdfm ) [ @xcite , @xcite , @xcite , @xcite , @xcite ] . while these approaches are appealing in the context of our case study , they do have some drawbacks . importantly ,
interpretation of component parameters is less straightforward under the ddpm , and the sdfm typically requires relatively long time series [ @xcite ] .
an alternative that we consider here is the use of informative priors at each epoch in a finite parametric mixture setting , where the information required at each epoch is obtained from neighbouring epochs .
this has appeal both in terms of a general bayesian learning framework and in terms of interpretability of the mixture components and weights , which is important in our application .
moreover , while some of the methods developed for mixture models in the spatial setting [ e.g. , @xcite ] can potentially be adapted for use in a time series setting , the influence or choice of informative priors in a time series framework and the implications in different data environments have largely not been examined .    in this paper
we explore three different informative priors for estimation of mixtures where the data are highly correlated , and all parameters in the mixture are allowed to vary . different simulated data sets , with features similar to actual particle size distribution data , are used to highlight the influence of using informative priors and to identify situations where placing informative priors may not be beneficial .
the paper is structured as follows . in section  [ secpsddata ]
we briefly describe particle size distributions and provide an illustration with actual data .
in section  [ secmixture ] we outline the finite parametric mixture model setup for a single time point and then outline the three approaches to estimation of a mixture model at multiple time points .
section  [ secres ] presents results on the performance of the approaches on several simulated data sets and actual data , and we conclude in section  [ secdiscussion ] with some discussion and possibilities for further work .
-axis : @xmath0 ) ] . the black ( overall ) and red ( components ) lines show the inferred density from estimation using . ]
figure  [ figplotexamplefit ] shows an example of particle size distribution data for one measurement or time point .
the histogram shows the number of particles @xmath1 per cubic centimeter binned by particle size , with the horizontal axis representing the natural logarithm of the particle diameter in nm [ @xmath2 .
the histogram is normalised , so that its total area equals 1 .
-axis and ( bottom ) @xmath3-axis is on the @xmath4 scale [ i.e. , @xmath5 . ]
because aerosol particles are charged , their size can be determined from their electrical mobility [ @xcite ] and a common instrument that utilises this principle is the differential mobility particle sizer ( dmps ) [ @xcite ] .    in this study
we present , as an example , the aerosol particle evolution before , during and after a new particle formation event at a boreal forest in southern finland ( figure  [ figplotoneday ] ) .
this data set was selected as it provides a wide ranging representation of modes for particle size distributions [ @xcite ] . because aerosol particles are governed by formation and transformation processes
, they tend to form well distinguishable modal features .
for example , during background conditions in the boreal forest the particle number size distribution of fine aerosols [ diameter @xmath62500 nm ( @xmath4 , 7.82 ) ] is bimodal : an aitken mode [ below 100  nm ( @xmath4 ,  4.60 ) ] and an accumulation mode ( over 100  nm ) . during a new particle formation event a new particle mode , which is commonly known as a nucleation mode ,
is formed in the atmosphere with geometric mean diameter below 25 nm ( @xmath4 , 3.22 ) . however , in the urban atmosphere aerosol particles are more dynamic because of the different types and properties of sources of aerosol particles and may show more than three modes .
typically the number concentrations of aerosol particles in the urban background can be as high as @xmath7  @xmath8 and close to a major road they often exceed @xmath9  @xmath8 [ @xcite ] .
in this section we briefly describe the mixture model , outline a  two stage approach to estimation of parameters over time , and describe three types of priors for temporal evolution of the parameters .
the density of data ( @xmath10 ) at a given time period may be represented by a finite parametric mixture model @xmath11 where @xmath12 is the number of components in the mixture , @xmath13 represents the probability of membership of the @xmath14th component ( @xmath15 ) , and @xmath16 ) is the density function of component @xmath14 which has parameters @xmath17 .
let @xmath18 indicate the observed data index .
as component membership of the data is unknown , a computationally convenient method of estimation for mixture models is to use a hidden allocation process and introduce a latent indicator variable @xmath19 , which is used along the lines of a missing variable approach to allocate observations @xmath20 to each component [ @xcite ] .    in this paper
we adopt the common assumption of fitting normal distributions to aerosol particle size distribution data [ @xcite ] .
as psd data are often measured with a definite lower and upper bound for the size of the particles , we introduce a slight modification and assume that the data follow a  truncated normal distribution .
as is commonly assumed , we take the data ( @xmath10 ) to be the @xmath4 of particle diameters ( nm ) , and the parameters ( @xmath17 ) for each component are the mean ( @xmath21 ) and variance ( @xmath22 ) .
the number of components @xmath12 is also assumed to be unknown .    in the first stage of the temporal analysis , for each time period we implemented a reversible jump markov chain monte carlo ( rjmcmc ) algorithm [ @xcite ] .
although this algorithm is easily fit at a single time point , the use of rjmcmc for mixture models with temporal data requires significant preprocessing with respect to mixing coverage and convergence , as well as postprocessing to provide adequate summary statistics and between time component mapping .    as an alternative
, we considered a two - stage approach . in the first stage ,
the number of components was estimated at each time point using rjmcmc . in the second stage , we fixed the number of components ( @xmath12 ) to the maximum observed at any time point and independently estimated the parameters of the mixture model ( @xmath23 , @xmath24 and @xmath25 ) for each time point using a mcmc sampler algorithm .
details of the mcmc scheme used for the different cases are given below . as we do not observe all of the components in every time point , we allow component weights to be `` effectively '' zero [ @xmath26 if required [ for details on the asymptotic behaviour of the posterior distribution using this approach see @xcite ] .
estimation of parameters of the components that are effectively `` empty '' under this criterion will then essentially be governed by their respective prior information . for the results to follow in section  [ secres ] we
thus only plot the parameters of components which are not `` empty '' .
priors for the first stage of the analysis were @xmath27 where @xmath28 , @xmath29 , @xmath30 , @xmath31 , @xmath32 , @xmath33 , @xmath34 are hyperparameters .
for the second stage , priors were @xmath35 where again @xmath36 and @xmath37 are hyperparameters , detailed below .
the prior for @xmath23 and @xmath38 could alternatively be decoupled and expressed as in stage  1 , but we did not see a noticeable difference in the results ( section  [ secres ] ) using either form . for the independent prior case , we use uninformative priors for @xmath23 , @xmath24 and @xmath25 .      in the second stage ,
four priors were considered for linking parameter values ( @xmath39 ) over time .
the first of these was the independent prior , in which the correlated nature of the data was ignored completely and parameters were independently estimated at each time point .
the second , third and fourth were termed the `` informed prior '' , `` penalised prior '' and `` hierarchical informed prior '' , as described below .      in this approach
we use the information provided from the previous time period as prior information for the current period . for the main results we focus on a simple case where posterior estimates from the previous period are used as prior information for the current period .
we do this to illustrate the influence of a simple prior specification on the posterior estimates of parameters  ( @xmath40 ) .    in the case of a mixture model using gaussian distributions , we have three parameters ( @xmath23 , @xmath24 and @xmath25 ) for which we could utilise available prior information to aid in estimation .
preliminary investigation indicates that all three parameters are likely to show strong evidence of autocorrelation , so here we examine the effect of smoothing on each of these parameters .    for @xmath41
, we allow @xmath42 in equation  ( [ eqnprior ] ) to reflect prior information about @xmath43 .
thus , we set @xmath44 , where @xmath45 is the mean of the number of observations allocated to component @xmath14 in the previous time period , and @xmath17 is fixed at some value .
an alternative is to impose a distribution on @xmath40 , say , @xmath46 [ or @xmath47 ) ] , but we do not present the results for this approach in this paper .
for the specification of prior information for @xmath23 and @xmath24 , we set @xmath48 , @xmath49 and @xmath50 [ to ensure @xmath51 is centred on @xmath52 and increase the value of @xmath53 from the value set for the independent case to reflect the degree of dependency for these parameters from the previous period .      in this approach we base the priors at time @xmath54 on the aggregated information at all other time periods .
this can be achieved by employing a reparameterisation of the prior to reflect the degree of dependency between parameters .
@xcite proposed a prior for @xmath25 ( in a different context ) which can be used in our setting to downweight large changes in probabilities in successive time periods .
let @xmath55 , then @xmath56 is defined as @xmath57 where smaller values of @xmath58 imply greater smoothing .
a potential advantage of using information about estimates both forwards and backwards in time is the additional information this may provide to guide parameter estimates in the current period .
this may be most useful if large changes in the parameter estimates occur for single periods of time .
for the purposes of comparison with section  [ secsimulateddata ] , we compare the results of using a similar formulation for @xmath25 in the informed prior approach ( without smoothing on @xmath23 and @xmath24 ) .    thus , prior distributions @xmath59 and @xmath60 are set as for the independent approach [ equation  ( [ eqnprior ] ) ] .    for this formulation
, we sampled from the posterior distribution of @xmath25 using a  rejection sampling approach outlined in appendix  [ app1 ] .      in this approach
an informative prior is placed at two different levels .
the aim of allowing for different levels is to provide flexibility to the form in which prior information is given in the model .
this flexibility may be needed in cases where the correlation structure can vary greatly over time : instead of imposing a smoothing structure directly on strongly varying parameters , we can provide a less restrictive smoothing through the hyperparameters .    for the hierarchical approach
, we will focus on parameters @xmath23 and @xmath25 as they are the main parameters of interest for the psd data ( see section  [ secrealdata ] and the ) . the hierarchical approach for @xmath23
is specified as @xmath61\\[-9pt ] \phi_{jt } & \sim&\mathcal{n } \bigl(\phi_{j , t-1 } , \varepsilon^{(s)}_{\phi } \bigr),\nonumber\end{aligned}\ ] ] where @xmath62 and @xmath63 are scalars , reflecting the variability of @xmath64 and @xmath65 , respectively . under this formulation ,
@xmath23 is used to estimate the mixture distribution at the level of the data , and @xmath58 represents the underlying correlation of @xmath23 over time [ assuming in this case an ar(1 ) process ] . in this setting
, we can interpret the ratio @xmath66 as reflecting the amount of information we have about the underlying behaviour ( signal ) of @xmath23 in comparison to estimates at the level of the data ( noise ) .
for the first time period ( @xmath67 ) , we set @xmath68 . for estimation of @xmath23 and @xmath58 , we use a gibbs sampling scheme . for details see appendix  [ app2 ] .    for the parameter @xmath25 ,
the dirichlet prior used in equation  ( [ eqnprior ] ) for the independent model and the informed prior approach is a very common prior for discrete probabilities . a natural extension to the dirichlet prior with a temporal component is to use its representation in terms of a gamma distribution .
however , the inflexibility of the gamma distribution makes it difficult to construct a temporal structure to the dirichlet prior .
an alternative formulation of the dirichlet in terms of the beta distribution does not appear to provide greater flexibility .
another alternative is to use a logistic - normal prior for @xmath25 , where @xmath69\\[-8pt ] \lambda_{jt } & = & \frac{\exp(w_{jt})}{\sum_{j=1}^{k-1}{\exp(w_{jt})}}\nonumber\end{aligned}\ ] ] and where @xmath70 is the mean value ( number of particles ) at time period @xmath54 .    using this functional form
, the parameterisation of @xmath25 in terms of a multivariate normal distribution allows for a suitably flexible form in which to explore a hierarchical structure for this parameter .
such flexibility , in comparison to the dirichlet distribution , has been investigated in a hierarchical approach for pooling of estimates across different sampling units [ @xcite ] .    in a hierarchical
setting and similar to the model used for @xmath23 , we can further say that @xmath71\\[-8pt ] \gamma_{jt } & = & \frac{\exp(x_{jt})}{\sum_{j=1}^{k-1}{\exp(x_{jt})}},\nonumber\end{aligned}\ ] ] where @xmath72 and @xmath73 reflect the variability of @xmath74 and @xmath70 , respectively .
analogous to the above discussion for @xmath23 , under this formulation the parameter @xmath25 is used to estimate the mixture model at the level of the data , and @xmath75 represents the underlying or smoothed behaviour of @xmath25 over time , which may be prone to large fluctuations from the data .    for the simulation results and actual data to follow ,
we specify the diagonal entries of @xmath72 and @xmath73 , and fix off - diagonal entries to be zero . for comparability with the hierarchical approach for @xmath23 , and using similar notation for the smoothing parameters , we specify @xmath76 and @xmath77 .
the interpretation of @xmath78 and @xmath79 is then the same as before , but this time in terms of @xmath25 .    for estimation of @xmath25 and @xmath75
we use a gibbs sampling scheme with a metropolis hastings step . for details see appendix  [ app2 ] . for identifiability
both @xmath74 and @xmath70 are @xmath80 dimensional , and @xmath81 ( with the same identification used for  @xmath75 ) .    in practice
, it may be difficult to specify _ a priori _ the parameter values for @xmath78 and @xmath79 , as little information about the variability of the parameters for the mixture components may be known .
estimation of these parameters also requires a choice to be made about the degree of smoothing required .
for the purposes of this paper we focus on specifying the parameter values for @xmath78 and @xmath79 , and explore briefly the effect on the results of varying these values . in the discussion we talk about this issue further .
for now , one approach to specifying @xmath78 may be to use the results from a rjmcmc approach used in the first stage , estimate or explore the variability of @xmath23 or @xmath25 over time , and then use this information to set the parameter values for @xmath78 .
the parameter value for @xmath79 could be set as a smaller multiple of  @xmath78 , and be varied to assess the influence of the results . using the information from the results of the independent approach , for a fixed upper bound @xmath12
could also be used .
although there is extra computational time involved in running either approach in the first stage , such a strategy may prove useful in order to assess the influence of different prior information on the results .      where component parameters are themselves the subject of the analysis , an important and commonly encountered issue in bayesian mixture modelling relates to the labelling of these parameters during the mcmc run .
as the likelihood of a mixture is by definition multimodal , using exchangeable or noninformative priors can ( and should ) result in parameters moving freely over the parameter and component labelling space during sampling .
( in theory , for a @xmath12 component mixture , @xmath82 permutations of the labelling of the parameters are possible . ) estimation of functionals of these parameters ( conditional on labelling ) at the end of sampling is thus then problematic . several empirical approaches to deal with this issue ( commonly called `` label switching '' ) have been proposed in the literature [ @xcite ( @xcite ) , @xcite , @xcite ( @xcite ) , @xcite , @xcite , @xcite , @xcite ] , generally by relabelling parameters in proximity to one of the @xmath82 modal regions during the run [ e.g. , @xcite ] or at the end of sampling [ e.g. , @xcite ] .    under exchangeable or noninformative priors
the main reason for label switching relates to a lack of identifiability of the mixture model ( particularly with respect to enforcing a unique labelling ) . given two component parameter sets @xmath83 and @xmath84 , a finite mixture model is weakly identifiable if at least one element of @xmath83 and @xmath84 differs . for practical purposes then
, the element that differs can be used to enforce a unique labelling . for exchangeable priors , in the sense
that priors for @xmath83 and @xmath84 are the same , it is clear that there is a lack of identifiability .    in the case of using informative priors on @xmath83 and @xmath84 ,
the issue of model identifiability is less clear .
although the use of different informative priors can help to separate @xmath83 and @xmath84 , in practice , it will depend on the strength of the informative prior to separate at least one element of each parameter set .
to our knowledge , there has been very little theoretical investigation of this issue . in practice
, one can assess whether parameters are well separated by analysing the path of parameters over the sampling run and/or from plots of marginal densities .
however , only part of the picture may be revealed by doing this .
first , a relatively long sampling run is needed to allow the parameter to fully explore the space ( perhaps many thousands of iterations ) . in the case of gibbs sampling , the sampler
may also become trapped in one of the modal regions [ @xcite ] .
apart from the case of using a fully exchangeable prior , this second case is more difficult to identify in practice , as it can be unclear whether the gibbs sampler is truly trapped or has found a uniquely labelled parameter space .
a pragmatic solution could be to start the sampler from different values , although good starting locations can be difficult to determine in high - dimensional space .
alternative ( and often more involved ) solutions could be to : reparameterise and change the conditioning [ @xcite ] ; use tempering to facilitate more exploration [ @xcite ] ; and/or modify the gibbs sampling proposal and acceptance [ @xcite , @xcite ] .    for both the independent and informed prior approaches outlined previously , it is possible to relabel the output using existing empirical approaches . in these cases ,
the posterior is updated sequentially across time and relabelling can take place during or at the end of each time period . in the results to be presented , we used the _
maximum a posteriori _ ( map ) estimate to select one of the @xmath82 modal regions and chose either a distance - based measure on the space of parameters [ @xcite ] , or on the space of allocation probabilities [ @xcite , @xcite ] to relabel parameters in proximity to this region .    in the case of the penalised and hierarchical informed priors , a joint posterior that includes all components and all parameters over time is used and sampling updates occur globally over time .
relabelling of sampling output thus necessitates a permutation of labels for all time points together .
such a joint approach is similar in spirit to the viterbi algorithm approach used in state space models [ @xcite ] , but the joint posterior in our case is different and would require further work outside the scope of this paper .    using informative priors that are incompatible with the data can also force too many distinct components and lead to overfitting , which can in some cases lead to label switching .
the potential for label switching could arise if two components with similar parameters are fitted where one component would suffice under the true model . as before , the similarity of the parameters could result in a lack of identifiability of the model . in using an informative prior
there is also a model choice issue in terms of selecting the best model for the data , where `` best '' is defined , for example , as the most parsimonious model in terms of the smallest value of @xmath12 . in practice ,
discrimination between the choice and use of informative priors can be made by fitting several mixture models to the data .
point - process representations of the estimated parameters from using a noninformative prior ( such as figure  [ figrjplotmeans ] ) can also offer a good visual guide as to the potential range and scope of the parameter space .
aerosol particle measurements are commonly recorded in the form of a number of distinct particle size ranges , or channels , the size and number of the channels being governed by the type and setup of the measurement instrument .
for example , in the sampled data from hyytil ( see figure  [ figplotexamplefit ] and section  [ secrealdata ] ) , we observed 32 distinct size partitions ( bins ) covering the range from 3  nm to 650  nm .    such coarsening of the data created by binning has an impact on density estimates and in a mixture context the number of components required to adequately model the data [ @xcite ] . to address this
, we add another step in the gibbs sampler in which we simulate a new latent variable ( say @xmath3 ) which is drawn from the believed underlying density of the data ( @xmath10 ) , in this case the fitted mixture model at current estimates of the parameters , at each iteration of the gibbs sampler .
as the sampling takes place within each bin , the simulation of the latent variable essentially involves sampling from a truncated normal distribution for which there are a number of proposed approaches .
for computational efficiency we used the slice - sampling approach of @xcite . for details of the approach and of the gibbs sampler
see @xcite . as the number of observations within each bin ( and hence the latent sample size ) is quite large , an extra step can be added after the latent variables are simulated in which the samples within each bin are divided into a number of sub - bins and computations in the gibbs sampler proceed based on the new binned data .
this can greatly speed up computations compared to using the full latent sample whilst reducing the coarsening of the data created by the original bins .
in general , we found comparable results to the full latent sample by using 3 additional sub - bins for each original bin ( in total 97 bins are used compared to the original 32 bins ) .
in this section we present and assess the results using simulated data and then present the results of applying the approaches to particle size distribution data from hyytil , finland .
we use the simulated data to test the impact of the different prior representations and the degree of smoothing .
we first use an informative and penalised prior only on the weights ( @xmath25 ) , and then assess the influence of using an informative prior on @xmath23 and @xmath24 in order to assess the influence of using prior information for each parameter separately .    for the independent , informed and penalised prior approaches
the results are based on 50,000 iterations with a burn - in period of 20,000 ( i.e. , the first 20,000 samples are discarded ) .
results using rjmcmc ( used in the case study ) are based on 200,000 iterations with a burn - in period of 100,000 .
convergence was assessed by visual inspection and using the gelman
rubin statistic [ @xcite ] .
we simulated data sets indicative of the type of behaviour of aerosol particle size distribution data observed at hyytil , a boreal forest site in southern finland ( smear  ii ) [ @xcite ] .
a particular feature of these particle size distribution data is both a growth in the mean and weight for some of the modes ( components ) and a decline in weight for others .
changes can also occur to the variance of the modes and at times they can follow a similar pattern to the weights over time .
we simulated data from two different cases . in the first case ( d1 ) , we simulated data which are highly correlated across time , a feature of particle size distribution data observed in practice for most time periods where measurements are commonly taken at small time intervals .
this data set was also simulated with parameter estimates where at times the mixture is not well identified ( component means and weights are not well separated ) . of interest in this setting
is the effect of using either the informed prior or penalised prior approach compared to the independent approach .    in practice
, it is quite common to observe sudden large changes in the number of particles measured which may persist for a number of time periods .
this is more often observed when there are relatively few particles for a particular size group , and more so for the smaller sized particles ( an example of this type of data is examined in section  [ secrealdata ] ) .
thus , for the second data set ( d2 ) we simulated data for the first component where the weight for the smaller sized particles is quite volatile . for this data set the mixture is well identified .
further details and results are available in the supplementary material [ @xcite ] .    for both cases ( d1 )  and
( d2 ) , we simulated data using three components on 32 distinct size partitions ( bins ) equally spaced ( on the @xmath4 scale ) covering the range from 3  nm to 20  nm in particle diameter ( on the @xmath4 scale 1  to  3 ) .
the sample size for each time period is 1000 and the total number of time periods was 100 .
further details of the sampling process for each case are provided in section  [ secd1 ] and the supplementary material [ @xcite , second data set ( d2 ) ] .
for the results to follow , except as specified otherwise , for the independent , informed prior and penalised prior approaches , we set the hyperparameters to be @xmath85 ; @xmath86 ; @xmath87 ; and @xmath88 , which were chosen to be weakly uninformative considering the range and size of the data .
for the independent and informed prior approaches , the original gibbs sampling output has been relabelled using a distance - based measure on the space of parameters [ @xcite ] and also ( as a check ) on the allocation space [ @xcite , @xcite ] .        as shown in figure  [ figsimoactualind ] ( black line ) , for the first data set we simulated data for the first component with a mean value increasing from 1.5 to 3.0 , and weight increasing from 0.1 to 0.6 and then decreasing to 0.3 , over time
. often a consequence of the growth in the first component is a decline in size and weight for the larger sized particles and this is reflected in the weight for the second component following an opposite pattern to the first component . for the third component ,
the weight increases from 0.1 to 0.3 over time .
the parameters @xmath23 and @xmath25 are simulated with some noise around the parameter values , and the sample size is 1000 .
( top ) , @xmath38 ( middle ) and @xmath25 ( bottom ) ] over time ( @xmath3-axis ) for independent approach : simulated data ( black ) ; independent ( red ) ; and 95% credible interval ( dotted line ) .
the columns represent the components ( components 1 to 3 ) . ]
figure  [ figsimoactualind ] also shows the results of using the independent approach .
we see that at times the parameter estimates for the independent approach deviate from the actual data .
figures  [ figsimoactualip ]  and  [ figsimoactualpp ] show the results for the informed prior and penalised prior compared to the actual data , respectively . in figure
[ figsimoactualip ] , the results show the effect of varying the degree of smoothing on @xmath25 for the informed prior using @xmath89 .
for the results of the penalised prior , we vary the degree of smoothing on @xmath25 using @xmath90 .
( top ) , @xmath38 ( middle ) and @xmath25 ( bottom ) ] over time ( @xmath3-axis ) for informed prior approach : simulated data ( black ) ; theta@xmath91 ( green ) ; theta@xmath920.8 ( blue ) ; theta@xmath93 ( purple ) ; and 95% credible interval ( dotted line ) .
the columns represent the components ( components 1 to 3 ) . ]     ( top ) , @xmath38 ( middle ) and @xmath25 ( bottom ) ] over time ( @xmath3-axis ) for penalised prior approach : simulated data ( black ) ; @xmath94 ( brown ) ; @xmath95 ( light blue ) ; @xmath96 ( dark green ) ; and 95% credible interval ( dotted line ) .
the columns represent the components ( components 1 to 3 ) . ]    in figure  [ figsimoactualip ] , we can see that the parameter estimates for @xmath25 for all three values of @xmath40 appear to closely follow the actual data , with the closest estimates to the actual data being for @xmath97 and 1.3 . as we are only using
an informed prior on the weights , the parameter estimates for @xmath23 and @xmath24 appear to be quite variable over time compared to the actual data .
however , the variability appears to be slightly less for these variables than for the independent approach ( figure  [ figsimoactualind ] ) and closer to the actual data over time .
of interest is the closeness of the parameter estimates of @xmath23 and @xmath24 for components 1 and 2 , which more clearly follow the true growth occurring in component 1 and the stability over time for component 2 compared to that observed for the independent approach .    in figure
[ figsimoactualpp ] , the parameter estimates for the penalised prior approach appear to deviate slightly from the actual data for components 1 and 2 . for the third component , the parameter estimates for the penalised prior approach follow the actual data with some noise .
overall , the results from the penalised prior approach are similar to the independent approach but with less variability over time .
we turn now to an assessment of the impact of using an informative prior for @xmath23 or @xmath24 over time .
we present results for the highly correlated data set , since this is the most sensitive of the simulated data as discussed above . here
we set @xmath98 , @xmath48 , @xmath99 and .
( top ) , @xmath100 ( middle ) and @xmath25 ( bottom ) ] over time ( @xmath3-axis ) for informed prior approach : simulated data ( black ) ; smoothing on @xmath23 ( orange ) ; smoothing on @xmath24 ( dark green ) ; and 95% credible interval ( dotted line ) .
the columns represent the components ( components 1 to 3 ) . ]    in figure  [ figsimoactualtpmusigma ] , the parameter estimates for the informative prior for @xmath23 appear to more closely follow the actual data than using an informative prior for @xmath24 .
although the parameter estimates for both approaches appear to be further away from the actual data than using an informative prior for @xmath25 , they do appear to be closer than under the independent approach .
figure  [ figcaseiisimosmlambdamu ] shows the results of using an informative prior on both @xmath23 and @xmath25 .
in this example , the results are similar to using an informative prior only on @xmath25 .
thus , depending on the objectives of the analysis , using an informative prior on both parameters may not be needed .
( top ) , @xmath38 ( middle ) and @xmath25 ( bottom ) ] over time ( @xmath3-axis ) for approaches : simulated data ( black ) ; independent ( red ) ; smoothing on @xmath23 and @xmath25 ( green ) ; and 95% credible interval ( dotted line ) .
the columns represent the components ( components 1 to 3 ) . ]      for the second simulated data set , where the weight for the smaller sized particles is quite volatile , the results of smoothing on @xmath23 , @xmath24 and @xmath25 for the informed prior and penalised prior suggests that large adjustments to one parameter ( e.g. , from volatility in some time periods ) are not supported unless compensatory measures can be taken by the other parameters .
in contrast to these results , we do not see any large compensatory adjustments being made to parameters by using a hierarchically based informative prior for @xmath75 .
details are available in the supplementary material [ @xcite ] .
the data set studied here was taken from a measurement site at hyytil , finland ; a plot of the measurements for the day selected is shown in figure  [ figplotoneday ] .
this particular day was selected as it shows a new particle formation event occurring , whereby a new mode of aerosol particles appears with a significant influx of particles ( as high as @xmath101  per  @xmath102 ) with a geometric mean diameter ( @xmath610 nm ) , growing later into the aitken ( 2590  nm ) or accumulation modes ( 100@xmath103 nm ) . in terms of a temporal mixture model setting
, we will be able to assess the performance of the four prior specifications outlined previously as new components are introduced and both a growth in the mean and weight for those components are observed .
the data from hyytil consists of measurements which were taken every 10  minutes ( 144 time periods ) and for each time period in the form of 32 distinct size partitions ( bins ) equally scaled ( on the @xmath4 scale ) covering the range of 3  nm to 650  nm ( on the @xmath4 scale 1 to 6.5 ) .    as outlined in section  [ secmixture ] ,
the first stage of our approach is to apply rjmcmc to each time period .
these results are then used to guide the choice of the number of components and initial parameter estimates for the second stage analysis , in which temporally correlated priors are used to model the evolution of the mixture parameters over time .
figure  [ figrjplotmeans ] shows the results of the first stage of the algorithm , with a plot of the posterior mean estimates for @xmath64 at each time point @xmath54 ( bottom panel ) , with the size of the circles indicating the corresponding weight @xmath104 .
the average number of components estimated with the highest probability over the day was four ; the minimum number of components was one , and the maximum number of components was five ( see top panel of figure  [ figrjplotmeans ] ) .
( posterior median ) from rjmcmc algorithm over time ( every 10 minutes ) ( hyytil ) .
( bottom ) : plot of posterior mean estimates for @xmath105 [ @xmath0 ) ] from algorithm over the same time period .
stage  1 of analysis for temporal evolution of parameters .
the size of the circles is proportional to the weight ( @xmath104 ) corresponding to @xmath64 . ]    for the second stage , we fixed the number of components to be five with the initial mean values equally spaced across the range of possible diameter values [ @xmath106 and @xmath107 .
figure  [ figdm512hmind ] shows the results of using the independent approach ( original output has been relabelled using a distance - based measure on the space of parameters [ @xcite ] ; similar results were obtained by relabelling on the allocation space [ @xcite , @xcite ] ) .
[ ( top )  @xmath0 ) ] , @xmath25 ( middle ) and particle count ( per @xmath108 ) ( bottom ) .
stage  2 of the analysis for the evolution of parameters .
measurements taken every 10 minutes .
colours indicate the components to which parameter estimates belong .
( the parameter estimates for the first component are black , parameters for the second component are red , for the third component they are green , etc . ) note : only components where @xmath109 are plotted . ]    due to the large number of particles ( per @xmath108 ) observed over the day ( see figure  [ figplotoneday ] ) , the results of estimation using the informed prior approach for @xmath23 and/or @xmath25 are very similar to the results of the independent approach , so are not shown here .
as we are interested in the effect of using an informative prior in this context , we rescale the number of particles by a factor of @xmath110 and assess the results . the median number of particles over the course of the day is then 6893 , reaching a maximum of 17,740 . using this rescaling of the data ,
the results of the independent approach are the same as shown in figure  [ figdm512hmind ] .
figure  [ figdm512hmlambda ] shows the results of estimation using the informed prior approach for @xmath23 and @xmath25 . for these results , @xmath111 , @xmath112 and @xmath48 , which provides for a moderately informative prior across time .
similar results were obtained using the hierarchical model with similar strength of prior information .
of interest to note is that in the original output we did not see any evidence of label switching within the gibbs sampling runs .
although there is some evidence of instability between 12:00 and 13:00 for the weight ( @xmath13 ) of two of the components ( newly formed and background particles ) , this appears to be due to instability in the data ( see figure  [ figrjplotmeans ] ) , with the means of these components remaining reasonably well separated during this period .     and @xmath25 .
posterior mean estimates for @xmath23 [ ( top )  @xmath0 ) ] , @xmath25 ( middle ) and particle count ( per @xmath108 ) ( bottom ) and 95% credible interval ( dotted line ) .
stage  2 of the analysis for the evolution of parameters .
measurements taken every 10 minutes .
colours indicate the components to which parameter estimates belong .
( the parameter estimates for the first component are black , parameters for the second component are red , for the third component they are green , etc . ) note : only components where @xmath109 are plotted . ]    compared to the results from the independent approach ( figure  [ figdm512hmind ] ) , the results from the informed prior approach for @xmath23 and @xmath25 suggest a clear pattern for both over the course of the day . in particular , and of interest to aerosol physicists , is the clear growth pattern shown for the mean of the first component representing the smaller sized particles .
the results indicate that these particles grow from approximately 1.40 ( 4  nm ) to 3.0 ( 20  nm ) in diameter on the natural @xmath4 scale ( or in nanometers ) .
the path of the parameters for the first component ( black ) is less clear using the independent approach , principally as a result of the newly formed smaller particles merging into the component representing the background particles ( green ) .
there is also some evidence of instability in the labelling _ across time _ , which is natural given the absence of temporal information in the prior .
we note that different representations of this data are possible depending on prior assumptions .
if we relax the assumed moderate correlation of the new and background particles , then there is less separation between the component associated to the newly formed particles and to that of the background particles , similar to the results we see for the independent approach .
ideally , more information is needed to model these data ; this could take the form of greater assumptions about the underlying stochastic process and the possible inclusion of external factors ( e.g. , meteorological data ) impacting / influencing the observed process .
we discuss this issue more in the following section .
in this paper we explored the problem of estimating bayesian mixture models at multiple time points . under different situations , approaches that employ information about neighbouring time points compared favourably to results based on an independent approach . by including additional temporal information about parameters for correlated time periods
, we may be able to better identify individual components at each time point . as an aid for inference
, we may also be able to obtain smoother parameter estimates over time and from this be able to clearly establish patterns or identify anomalies from the data .
the results highlight a number of observations about mixture representations at multiple time points .
first , analysis of the evolution of parameters of a mixture over multiple time points highlights the large degree of dependency that exists between component parameters .
changes to a parameter in one component may flow on to the parameter in a nearby component .
depending on the context of the study , we can anticipate this dependency to be more readily apparent for the weight parameters , but we found similar dependencies to exist for other parameters .
the second is the need to be mindful that the same parameter in one component may have a different correlation structure over time to the same parameter in another component . in the context of particle size distribution data , we often observed greater volatility in estimates for the smaller particles compared to the larger sized particles and so at times the correlation structure of the parameters between these respective components appeared to be quite different .
a possible effect of using informative priors in this context is to impose a prior not supported by the data or to impose a temporal correlation structure where such a structure does not exist , and thereby cause unnecessary adjustments to other parameters .
we observed this most clearly in the results from the simulated data where at times the data was quite noisy .
for this data set , using an informative prior for a parameter , which supported large adjustments away from the actual data , resulted in large compensatory adjustments being made not only by other parameters within the same component , but also to parameters in neighbouring components .
the easy solution may be to use an appropriate correlation structure for components , but of course this may not always be known _
a priori_.    a further result of the dependency that can exist between parameters of components and within component parameters is that the inclusion of correlation information to aid in the identifiability of the mixture may not be required for all or , alternatively , all components . in the context of a mixture with a small number of components
, we may only need to provide more information about one parameter for an influential component in order to separate out the influence of competing components .
this result will also be useful if the correlation structure for one parameter or parameters for one component are more readily known . in the context of a mixture of gaussians
, we generally found that an informative prior was only needed on @xmath23 or @xmath25 or possibly both .
this result could well be context specific and influenced by any reliance on the means for defining ( in terms of size ) and ordering of components .
the choice of which parameter to use more information may also be guided by whether it is a parameter of interest for inference as demonstrated in analysis of the case study where most interest was in the behaviour of both @xmath23 and @xmath25 over time . in this case , and in general , one must be careful in the analysis of selected parameters , as it can largely be a conditional analysis in view of the behaviour of other possible cross - correlated parameters within the same component and between components .    in the hierarchical informed prior approach
the influence of the informative prior at the two levels was specified by parameters @xmath78 ( low level ) and @xmath79 ( high level ) , and the values assigned to these parameters are critical in carrying information about the correlation structure of the parameter of interest . in this paper , we decided to choose parameter values based on prior belief in the correlation structure of the data ; alternatively , these parameters could be estimated . to this effect ,
a number of approaches are available for estimation [ @xcite , @xcite ] . however , in order to estimate @xmath78 and @xmath79 , we still face a choice as to the degree of penalisation or smoothing of the parameter in light of the apparent variability in the data . this is a common issue in temporal and spatial modelling in general .
while many of the above difficulties may seem to be avoided if smoothing approaches are applied retrospectively on parameter estimates from an independent mixture model , this type of analysis may largely ignore the true mapping of components or the path of parameters over time . from the results of the simulated data ,
the large degree of dependency that we observe between the parameters of a mixture over time suggests that including temporal information to better identify one of the parameters at a single time point can flow on to affect other parameters .
this could change inference about both the mixture representation at a point in time and also the behaviour of mixture parameters over time .
in general , one of the potential difficulties in using an informative prior approach to smooth parameter estimates over time is the variable degree of influence the prior may have in the posterior .
if the primary objective is to obtain smoothed parameter estimates over time , larger sample sizes and noisiness of the data at times may warrant increasingly restrictive priors .
in such cases where the objective might be to downplay the influence of the data , a number of alternative approaches to increase the influence of prior information can be used [ @xcite ] . in all cases , it is valuable to undertake a sensitivity analysis in order to assess the effect of the prior . such an analysis should include the independent prior as a baseline comparison .
a further limitation of the approach outlined is that it is computationally expensive .
most of this expense is experienced in the first stage of the analysis ( which can be skipped in the presence of good prior knowledge of the parameter space ) .
for estimation of psd data over one day using 144 time points , the running time of the rj approach with 200,000 iterations was approximately 3 hours using an intel centrino 2 processor 2.80  ghz . in comparison , the second stage approach using 50,000 iterations took approximately one hour .
such computational expense quickly becomes burdensome if analyses is required for several days or , indeed , several weeks .
of course , the use of the first stage for subsequent days may not be required , considerably reducing the computational time involved .
although we have focussed on developing a hierarchical approach for parameters @xmath23 and @xmath25 , we could equally apply the same approach to consider estimation of  @xmath24 .
such an approach may be to consider a half-@xmath54 distribution which has previously been used in similar hierarchical settings [ @xcite ] .
the hierarchical approach considered here can be readily generalised to include covariates .
moreover , through the flexibility of assuming a logistic normal distribution on the weights , we can better explore and estimate transitory movements between components .    in some situations it may be of interest to combine components and allow components to share a common grouping .
this could be of interest where some components are only needed to account for the skewness of a larger component or to allow an analysis based on a mixture representation with fewer components of most interest .
although this grouping of components could be undertaken retrospectively , it may also be interesting to see the effects such a grouping has upon estimation of the parameters and their evolution over time .    for estimation of aerosol particle size distributions ,
the dynamics of the aerosol process and the complexity of the influences on particle concentration and size demand the use of approaches which utilise as much information from the data as possible . to this end
, the inclusion of temporal information may be helpful .
in this section we outline the rejection sampling algorithm for @xmath25 proposed by @xcite for the penalised prior approach :    prior @xmath113    posterior @xmath114\\[-16pt ] & & \qquad \propto\prod_{j=1}^{k } \biggl\ { \prod_{t=1}^{t } \operatorname{dirichlet}(m_{jt}+1)\mathrm{i}(\lambda_{jt } ) \biggr\ } \exp\biggl(-\frac{1}{\phi}\sum_{t=2}^{t } \|\underline{\lambda}_{t}-\underline{\lambda}_{t-1 } \|^{2 } \biggr).\nonumber\end{aligned}\ ] ]    @xcite suggest sampling @xmath115 from a @xmath116 , @xmath117 distribution and accepting when @xmath118 ( @xmath119 ) , where @xmath120\bigr)\nonumber\end{aligned}\ ] ] and @xmath121\bigr),\nonumber\end{aligned}\ ] ] where @xmath122 @xmath123 and @xmath124 . here
@xmath125 is an indicator function equal to  1 when @xmath126 ^ 2 $ ] and 0 otherwise .
_ hierarchical model for @xmath23 _    update @xmath127 , @xmath128 , @xmath25 , @xmath38 as in the independent approach .
update @xmath58 and @xmath23 by sampling from the conditionals , @xmath129 _ hierarchical model for @xmath25 _    update @xmath127 , @xmath128 , @xmath23 , @xmath38 as in the independent approach .
update @xmath130 by sampling from the conditional , @xmath131 where @xmath132 .
update @xmath133 using a metropolis hastings step
.    sample from @xmath134 , where @xmath135 and @xmath136 is the variance of the proposal .
let @xmath137 and for @xmath138 , @xmath139 , where @xmath140 is the mean number of observations allocated to component @xmath14 in the previous time period ( under the independent approach ) .
the authors gratefully acknowledge funding from the australian research council ( arc ) as part of two arc discovery projects and the arc centre of excellence in complex dynamic systems and control .
the authors are also grateful for helpful discussions with christian p. robert and tony pettitt in early stages of this work , and to comments from the editor , associate editor and referees .