relations between social actors can be studied by methods of social network analysis [ e.g. , wasserman and faust ( @xcite ) ; carrington , scott and wasserman ( @xcite ) ] .
examples are friendship between pupils in a school class or alliances between firms .
a basic data structure for social networks is the directed graph or digraph , where the actors are represented by the nodes , and the arcs between the nodes indicate the social ties . social network analysis traditionally has had a focus on rich description of network data , but the recent development of methods of statistical inference for network data [ e.g. , airoldi et al .
( @xcite ) ; hunter and handcock ( @xcite ) ] has the potential of moving this field toward a wider use of inferential approaches .
longitudinal studies are especially important for obtaining insight into social networks , but statistical methods for longitudinal network data that are versatile enough for realistic modeling are only just beginning to be developed .
this article considers repeated observations of a relation , or network , on a given set of actors @xmath0 , observed according to a panel design , and represented as a sequence of digraphs @xmath1 for @xmath2 , where @xmath3 are the observation moments .
the  nodes represent social actors ( which may be individuals , companies , etc . ) , and the node set @xmath4 is the same for all observation moments .
a digraph is defined here as an irreflexive relation , that is , a subset @xmath5 of @xmath6 , and when @xmath7 we shall say that there is an arc , or a tie , from @xmath8 to @xmath9 .
it often is realistic to assume that the social network has been developing between the observation moments , which leads to the assumption that the observations @xmath1 are realizations of stochastic digraphs @xmath10 embedded in a continuous - time stochastic process @xmath11 , @xmath12 .
holland and leinhardt ( @xcite ) proposed to use continuous - time markov chains , defined on the space of all digraphs with a given node set , for modeling social network dynamics even if the observations are made at a few discrete time points and not continuously .
continuous - time markov chains provide a natural starting point for modeling longitudinal network data .
wasserman ( @xcite , @xcite ) and leenders ( @xcite ) elaborated _ dyad - independent models _
, where the ties between pairs of actors ( _ dyads _ ) develop according to processes that are mutually independent between dyads .
this is not realistic for social processes , because dependence between the set of ties among three or more actors can be very strong , as was found already by davis ( @xcite ) who showed that a basic feature of many social networks is the tendency toward transitivity ( `` friends of my friends are my friends '' ) .
snijders and van duijn ( @xcite ) and snijders ( @xcite ) proposed so - called _ actor - oriented models _
, explained in the next section , which do allow such higher - order dependencies .
the  actor - oriented models are too complicated for the calculation of likelihoods or estimators in closed form , but they represent stochastic processes which can be easily simulated .
this was exploited by the estimation method proposed in the papers mentioned , which is a method of moments ( _ mom _ ) estimator implemented algorithmically by stochastic approximation [ robbins and monro ( @xcite ) ; also see , e.g. , kushner and yin ( @xcite ) ] .
this estimator usually performs well [ some empirical applications are given in de federico ( @xcite ) , van de bunt , van duijn and snijders ( @xcite ) , and van duijn et al .
( @xcite ) ] .
it is to be expected , however , that the statistical efficiency of the maximum likelihood ( _ ml _ ) estimator will be greater .
ml estimation also paves the way for likelihood - based model selection , which will be a marked improvement on existing methods of model selection .
this article presents an mcmc algorithm for approximating the ml estimator , combining and extending ideas in gu and kong ( @xcite ) , snijders ( @xcite ) , and koskinen and snijders ( @xcite ) , the latter of which proposed for this model a mcmc algorithm for bayesian inference .
section [ sec2 ] presents the model definition . the  algorithm for obtaining the ml estimator
is described in section [ sec3 ] .
section [ sec4 ] reports results of an empirical example and section [ sec5 ] presents a very small simulation study comparing the ml and mom estimators .
the  paper finishes with an algorithm for approximating the likelihood ratio test in section [ sec6 ] and a discussion in section [ sec7 ] .
we assume that repeated observations @xmath13 on the network are available for some @xmath14 . the  network , or digraph , @xmath5 will be identified with its @xmath15 adjacency matrix , of which the elements denote whether there is a tie from node @xmath8 to node @xmath9 ( @xmath16 ) or not ( @xmath17 ) .
self - ties are not allowed , so that the diagonal is structurally zero .
random variables are denoted by capitals . the  stochastic process @xmath11 , in which the observations @xmath1 are embedded , is modeled as being right - continuous .
various models have been proposed , most of them being markov processes of some kind .
we focus on actor - oriented models [ snijders ( @xcite ) ] .
tie - oriented models [ snijders ( @xcite ) ] can be treated similarly .
the  basic idea of actor - oriented models [ snijders ( @xcite ) ] is that the nodes of the graph represent social actors , who have control , albeit under constraints , of their outgoing ties ; and the graph develops as a continuous - time markov process ( even though it is observed only at @xmath18 discrete time points ) .
the  constraints are that ties may change only one by one , and actors do not coordinate their changes of ties .
thus , at any given moment , one actor @xmath8 may create one new tie or delete one existing tie , where the probability distribution of such changes depends on the current digraph ; excluded are simultaneous changes such as swapping one tie for another , or bargaining between actors over ties .
this constraint was proposed already by holland and leinhardt ( @xcite ) , and it has the virtue of splitting up the change process in its smallest possible constituents .
the  actor - oriented model is further specified as follows ; further discussion and motivation is given in snijders ( @xcite ) .    1 .
_ opportunities for change _
each actor @xmath8 gets at stochastically determined moments the opportunity to change one of the outgoing tie variables @xmath19 ( @xmath20 , @xmath21 ) .
since the process is assumed to be markovian , waiting times between opportunities have exponential distributions .
each of the actors @xmath8 has a _ rate function _
@xmath22 which defines how quickly this actor gets an opportunity to change a tie variable , when the current value of the digraph is @xmath5 , and where @xmath23 is a parameter . at any time point @xmath24 with @xmath25 , the waiting time until the next opportunity for change by any actor
is exponentially distributed with parameter @xmath26 given that an opportunity for change occurs , the probability that it is actor @xmath8 who gets the opportunity is given by @xmath27 the  rate functions can be constant between observation moments , or they can depend on functions @xmath28 which may be covariates or positional characteristics of the actors such as outdegrees @xmath29 .
a convenient assumption is to use an exponential link function , @xmath30    1 .
_ options for change _    when actor @xmath8 gets the opportunity to make a change , this actor has a permitted set @xmath31 of values to which the digraph may be changed , where @xmath32 is the current value of the digraph .
the  assumption that the actor controls his or her outgoing ties , but can change only one tie variable at the time , is equivalent to    [ ai ] @xmath33 where @xmath34 is the set of adjacency matrices differing from @xmath5 in exactly one element , @xmath35\\[-8pt ] & & \hspace*{14pt}\mbox { and } x_{hk } = x^0_{hk } \mbox { for all other } ( h , k ) \ } .\nonumber\end{aligned}\ ] ]    including @xmath32 in @xmath31 can be important for expressing the property that actors who are satisfied with the current network will prefer to keep it unchanged .
therefore , the usual model is @xmath36 .
this does not lead to identifiability problems because the ratio between not making a change and making a change is not a free parameter , but fixed by assumption ( [ choiceprob ] ) for the conditional probabilities of the new state of the network .
some alternatives are models with structurally impossible ties , where the impossible digraphs are left out of @xmath31 , and models where the actor is required to make a change whenever there is the opportunity , obtained by leaving the current element @xmath32 out of @xmath31 .
it is assumed that the network dynamics is driven by a so - called _ objective function _ @xmath37 that can be interpreted as the relative attractiveness for actor @xmath8 of moving from the network represented by @xmath32 to the network @xmath5 , and where @xmath38 is a parameter . under the condition
that the current digraph is @xmath32 and actor @xmath8 gets the opportunity to make a change , the conditional probability that the next digraph is  @xmath5  is modeled as @xmath39 where the summation extends over @xmath40 .
this formula can be motivated by a random utility argument as used in econometrics [ see , e.g. , maddala ( @xcite ) ] , where it is assumed that the actor maximizes @xmath37 plus a random disturbance having a standard gumbel distribution . assumption ( [ ai ] ) implies that instead of @xmath41 we can also write @xmath42 where the correspondence between @xmath5 and @xmath9 is defined as follows : if @xmath43 , @xmath9 is the unique element of @xmath4 for which @xmath44 ; if @xmath45 , @xmath46 .
this less redundant notation will be used in the sequel . thus , for @xmath21
, @xmath42 is the probability that , under the condition that actor  @xmath8 has the opportunity to make a change and the current digraph is @xmath32 , the change will be to @xmath47 with the rest unchanged , while @xmath48 is the probability that , under the same condition , the digraph will not be changed .
the  most usual models are based on objective functions that depend on @xmath5 only .
this has the interpretation that actors wish to maximize a function @xmath49 independently of `` where they come from . ''
the  greater generality of ( [ choiceprob ] ) , where the objective function can depend also on the previous state @xmath32 , makes it possible to model path - dependencies , or hysteresis , where the loss suffered from withdrawing a given tie differs from the gain from creating this tie , even if the rest of the network has remained unchanged .    various ingredients for specifying the objective function were proposed in snijders ( @xcite ) .
a linear form is convenient , @xmath50 where the functions @xmath51 are determined by subject - matter knowledge and available social scientific theory .
these functions can represent essential aspects of the network structure , assessed from the point of view of actor @xmath8 , such as @xmath52 they can also depend on covariates  such as resources and preferences of the actors , or costs of exchange between pairs of actors  or combinations of network structure and covariates .
for example , de federico ( @xcite ) found in a study of friendship between foreign exchange students that friendship formation tends to be reciprocal , with a negative parameter for forming indirect ties ( thus leading to relatively closed networks ) , and that friendships are more likely to be formed between individuals from the same region ( a covariate effect ) , but that reciprocation adds less to the tendency to form ties between persons from the same region than between arbitrary individuals ( negative interaction between covariate and reciprocity ) .    1 .   _ intensity matrix ; time - homogeneity _
the  model description given above defines @xmath11 as a continuous - time markov process with @xmath53-matrix or intensity matrix [ e.g. , norris ( @xcite ) ] for @xmath43 given by @xmath54 the  assumptions do not imply that the distribution of @xmath11 is stationary .
the  intensity matrix is time - homogeneous , however , except for time dependence reflected by time - varying components in the functions @xmath51 .    for the data - collection designs to which this paper is devoted , where observations are done at discrete time points @xmath55 , these time points can be used for marking time - heterogeneity of the transition distribution ( cf .
the example in section  [ sec6 ] ) .
for example , covariates may be included with values allowed to change at the observation moments .
a special role is played here by the time durations @xmath56 between successive observations .
standard theory for continuous - time markov chains [ e.g. , norris ( @xcite ) ] shows that the matrix of transition probabilities from @xmath10 to @xmath57 is @xmath58 , where @xmath53 is the matrix with elements  ( [ q ] ) .
thus , changing the duration @xmath56 can be compensated by multiplication of the rate function @xmath59 by a constant .
since the connection between an externally defined real - valued time variable @xmath60 and the rapidity of network change is tenuous at best , it usually is advisable [ e.g. , see snijders ( @xcite ) ] to include a multiplicative parameter in the rate function which is constant between observation moments @xmath60 but differs freely between periods @xmath61 . with the inclusion of such a parameter , the numerical values @xmath60
become unimportant because changes in their values can be compensated by the multiplicative rate parameters .
popular models for analyzing cross - sectional network data are exponential families of graph distributions such as the markov model of frank and strauss ( @xcite ) , generalized to the @xmath62 exponential random graph ( erg ) model by frank ( @xcite ) and wasserman and pattison ( @xcite ) , and elaborated and further specified by hunter and handcock ( @xcite ) and snijders et al .
( @xcite ) .
discrete - time markov chain models , as longitudinal models of this kind , were proposed by robins and pattison ( @xcite ) , krackhardt and handcock ( @xcite ) , and hanneke and xing ( @xcite ) .
there are a number of essential differences between these models and the actor - oriented model treated here , with respect to interpretation as well as statistical procedures .
when applying the actor - oriented model to a sequence of two or more repeated observations , these observations are embedded in a continuous - time model .
this has clear face validity in applications where changes in the network take place at arbitrary moments between observations .
singer ( @xcite ) gives an overview of the use of this principle for continuously distributed data .
this approach yields a major advantage over the cited longitudinal erg models : our probability model is defined independently of the observational design .
data from irregularly spaced observation intervals can be analyzed without the need to make any adaptations .
another advantage is that the dynamic process is defined parsimoniously as a function of its elementary constituents  in this case , the conditional probability of a single tie change .
the  random utility interpretation of ( [ choiceprob ] ) , discussed above , gives an interpretation in terms of myopic optimization of an objective function to which a random disturbance is added , and which can be used to represent a `` social mechanism '' that could have given rise to the observed dynamics .
the  discrete - time erg models constitute exponential families , which has the advantage that many standard techniques can be directly applied .
the  distribution of the continuous - time process @xmath63 for the actor - oriented model constitutes an exponential family , so for discrete observation moments our model can be regarded as an incompletely observed exponential family . for both types of model
, inference is computer - intensive and time - consuming because of the need to implement elaborate mcmc procedures .
the  definition of the model given above can be used directly to simulate data from the probability distribution , conditional on an initial state of the network .
this contrasts with models in the erg family , which can be simulated only indirectly , by regarding them as the stationary distribution of an auxiliary markov chain , and applying a gibbs or metropolis  hastings algorithm .
the  near degeneracy problem [ snijders et al .
( @xcite ) ] which plagues some specifications of erg models is , in practice , not a problem for the actor - oriented model .
this section presents an algorithm for mcmc approximation of the maximum likelihood estimate .
estimation is done conditional on the first observation @xmath64 .
this has the advantage that no model assumptions need to be invoked concerning the probability distribution that may have led to the first observed network @xmath64 , and the estimated parameters refer exclusively to the dynamics of the network .
the  algorithm proposed below can be sketched as follows . for each @xmath65(@xmath66 )
the observed data are augmented with random draws from the sequence of intermediate digraphs @xmath67 that could have led from one observation , @xmath68 , to the next , @xmath1 ( section  [ s_aug ] ) .
these draws can be simulated using a metropolis  hastings algorithm ( section  [ s_sim ] ) .
they are then used in the updating steps of a robbins  monro algorithm , following ideas of gu and kong ( @xcite ) , to find the solution of the likelihood equation ( section  [ s_appr ] ) .
these elements are put together in section  [ s_multi ] .      the  likelihood for the observed data @xmath69 conditional on @xmath64
can not generally be expressed in a computable form .
therefore , the observed data will be augmented with data such that an easily computable likelihood is obtained , employing the general data augmentation principle proposed by tanner and wong ( @xcite ) .
the  data augmentation can be done for each period @xmath61 separately and , therefore , this section considers only the observations @xmath64 and @xmath70 .
denote the time points of an opportunity for change by @xmath71 and their total number between @xmath72 and @xmath73 by @xmath74 , the time points being ordered increasingly so that @xmath75 .
the  model assumptions imply that at each time  @xmath71 , there is one actor , denoted @xmath76 , who gets an opportunity for change at this time moment .
define @xmath77 as the actor toward whom the tie variable is changed , and define formally @xmath78 if there is no change [ i.e. , if @xmath79 ) ] : @xmath80 given @xmath64 , the outcome of the stochastic process @xmath81 , @xmath82 completely determines @xmath83 .
the  augmenting data consist of @xmath74 and @xmath84 , @xmath82 , without the time points @xmath71 .
it may be noted that this differs from the definition of a sample path in koskinen and snijders ( @xcite ) in that the times in between opportunities for change are integrated out ; the reason is to obtain a simpler mcmc algorithm .
the  possible outcomes of the augmenting data are determined by the condition @xmath85 for all @xmath86 with @xmath87 .
the  stochastic process @xmath88 , @xmath89 will be referred to as the sample path ; the elements for which @xmath90 , although redundant to calculate @xmath70 from @xmath64 , are retained because they facilitate the computation of the likelihood .
define @xmath91 ; the digraphs @xmath92 and @xmath93 differ in element @xmath94 provided @xmath95 , and in no other elements .
the  probability function of the sample path , conditional on @xmath64 , is given by @xmath96 where @xmath97 is defined in ( [ wi ] ) and @xmath98 in and just after ( [ choiceprob ] ) .
denote the first component of ( [ full_like1 ] ) by @xmath99\\[-8pt ] & & \qquad = p_{\alpha , \beta}\bigl\{t_{r } \leq t_2 < t_{r+1 } \vert x^{(0 ) } , ( i_1 , j_1 ) , \ldots , ( i_r , j_r ) \bigr\ }   .\nonumber\end{aligned}\ ] ] conditioning on @xmath100 [ and not on @xmath70 ! ] ,
the differences @xmath101 are independently exponentially distributed with parameters@xmath102 . hence , under this conditioning the distribution of @xmath103 is the convolution of exponential distributions with parameters @xmath102 for @xmath104 . in the special case that the actor - level rates of change @xmath22 are constant , denoted by  @xmath105 , @xmath74  has a poisson distribution with parameter @xmath106 ; ( [ kappa1 ] ) then is given by @xmath107 in the general case where the change rates are nonconstant , the probability ( [ kappa1 ] ) can be approximated as follows .
denote by @xmath108 the density function of @xmath109 , conditional on @xmath110 , @xmath111 or , equivalently , on the embedded process @xmath112 the  distribution of @xmath103 is a convolution of exponential distributions and , therefore , the central limit theorem implies that the density function @xmath108 is approximately the normal density with expected value @xmath113 and variance @xmath114 hence , @xmath115 probability ( [ kappa1 ] ) now can be expressed as @xmath116 the  approximations are valid for large @xmath74 and @xmath117 , under boundedness conditions on the rate functions @xmath118 . the  first approximation in ( [ kappa2 ] ) is based on splitting the integration interval into two intervals @xmath119 and @xmath120 for a bounded but large @xmath121 ; the first interval then gives an asymptotically negligible contribution and on the second interval @xmath122 is approximately constant since @xmath123 .
the  second approximation uses that @xmath117 is large .
combining the preceding equations yields @xmath124\\[-8pt ] & & \qquad \approx \frac{1}{\lambda(\alpha , x^{(r ) } ) \sqrt{(2\pi\sigma ^2_\alpha ) } }    \exp\biggl ( \frac{-(t_2 - t_1 - \mu_\alpha)^2}{2\sigma^2_\alpha } \biggr )   .\nonumber\end{aligned}\ ] ] this shows that , for observed data @xmath125 augmented by the sample path , the likelihood conditional on @xmath64 can be expressed directly , either exactly or in a good approximation .      an mcmc algorithm will be used that finds the ml estimator based on augmented data .
several methods for mcmc maximum likelihood estimation have been proposed in the literature .
we shall use the markov chain stochastic approximation ( mcsa ) algorithm proposed by gu and kong ( @xcite ) and used ( in a slightly different specification ) also by gu and zhu ( @xcite ) .
this algorithm is based on the missing information principle of orchard and woodbury ( @xcite ) and louis ( @xcite)going back to fisher ( @xcite ) ; cf .
efron ( @xcite ) .
the  principle can be summarized as follows .
suppose that @xmath5 is observed , with probability distribution parameterized by @xmath126 and having probability density @xmath127 w.r.t . some @xmath128-finite measure . to facilitate estimation ,
the observed data is augmented by extra data @xmath129 ( regarded as missing data ) such that the joint density is @xmath130 .
denote the observed data score function @xmath131 by @xmath132 and the total data score function @xmath133 by @xmath134 .
it is not hard to prove ( see the cited literature ) that @xmath135 this is the first part of the missing information principle .
this equation implies that the likelihood equation can be expressed as @xmath136 and , therefore , ml estimates can be determined , under regularity conditions , as the solution of ( [ mle1 ] ) .
this is applied in situations where the observed data score function @xmath132 is too difficult to calculate , while the total data score function @xmath134 is computable . in our case , we condition on @xmath137 , so this is treated as being fixed ; we observe @xmath138 ; and between each pair of consecutive observations @xmath57 and @xmath10 the data are augmented , as discussed in the previous section , by the sample path that could have brought the network from  @xmath57 to  @xmath10 .
these sample paths combined for @xmath139 to @xmath18 constitute @xmath140 . the  following two subsections supply the additional elements for how the augmenting data are used .    in the mcsa algorithm of gu and kong ( @xcite ) ,
the solution of ( [ mle1 ] ) is obtained by stochastic approximation [ robbins and monro ( @xcite ) ; kushner and yin ( @xcite ) ] which is defined by the updating step @xmath141 where @xmath142 is generated according to the conditional distribution of @xmath140 , given , with parameter value @xmath143 .
the  sequence @xmath144 , called the gain sequence , consists of positive numbers , tending to 0 .
the  matrix @xmath145 is a suitable matrix .
it is efficient [ see kushner and yin ( @xcite ) ] to use a gain sequence tending to zero as @xmath146 for a @xmath147 , and to estimate @xmath126 not by the last element @xmath143 produced by the algorithm , but by a tail average @xmath148 . for fixed @xmath149 and @xmath150 ,
this average converges to the solution of ( [ mle1 ] ) for a wide range of positive definite matrices @xmath145 .
the  second part of the missing information principle [ orchard and woodbury ( @xcite ) ] is that the observed fisher information matrix for the observed data can be expressed as @xmath151\\[-8pt ] & = & e_\theta   \{d_{xv}(\theta ) \mid x = x \ } - \mbox{cov}_\theta   \ { s_{xv}(\theta ;   x , v ) \mid x = x \ } , \nonumber\end{aligned}\ ] ] where @xmath152 is the complete data observed fisher information matrix , @xmath153 expression ( [ fi2 ] ) can be interpreted loosely as `` information is total ( but partially unobserved ) information minus missing information . ''
this formula allows us to calculate standard errors .
the  mcsa method relies on monte carlo simulations of the missing data @xmath140 .
the  markov property allows us to treat all periods @xmath61 separately and , therefore , we simplify the treatment and notation here by treating only one of those periods , temporarily assuming that @xmath154 .
the  missing data then is the sample path @xmath155 which specifies the sequence of tie changes that brings the network from @xmath156 to @xmath157 .
the  advantage of having integrated out the time steps @xmath71with the use of the expressions ( [ kappa4 ] ) and ( [ kappa2])is that less random noise is introduced , and the simulated variable  @xmath140 is discrete rather than including a euclidean vector with a varying dimension @xmath74 , which would require a more complicated mcmc procedure .
the  set of all sample paths connecting @xmath64 and @xmath70 is the set of all finite sequences of pairs @xmath86 , @xmath158 , where for @xmath87 the parity of the number of occurrences of @xmath86 is given by ( [ x2 ] ) .
such sequences will be denoted by @xmath159 , and the set of all these sequences is denoted @xmath160 .
the  probability of the sample path conditional on @xmath161 is proportional to ( [ full_like1 ] ) , rewritten here as @xmath162 where @xmath163 is given by ( [ kappa4 ] ) or approximated by ( [ kappa3 ] ) , depending on the specification of @xmath118 .
draws from this distribution can be generated by the metropolis
hastings algorithm , provided that a proposal distribution is used which connects any two elements of @xmath160 .
such a proposal distribution will now be given , denoting the proposal probabilities by @xmath164 and the target probabilities , which are proportional to ( [ full_like2 ] ) , by  @xmath165 .
then the acceptance probabilities in the metropolis  hastings algorithm , for a current state @xmath166 and a proposed state @xmath167 , are @xmath168    a proposal distribution is used that consists of small changes in @xmath166 .
the  construction of the proposal distribution was based on the considerations that the proposal distribution should mix well in the set of all sample paths , and the metropolis
hastings ratios in ( [ mh ] ) should be computable relatively easily .
this led to proposal distributions consisting of the following types of small changes in @xmath166 :    1 .   _
paired deletions_. of all pairs of indices @xmath169 with @xmath170,@xmath171 , one pair is randomly selected , and @xmath172 and @xmath173 are deleted from @xmath166 .
paired insertions_. a pair @xmath174 with @xmath87 is randomly chosen , two indices @xmath169 are randomly chosen , and the element @xmath86 is inserted immediately before @xmath175 and before @xmath176 .
single insertions_. at a random place in the path ( allowing beginning and end ) , the element @xmath177 is inserted for a random @xmath178 .
single deletions_. of all elements @xmath179 satisfying @xmath180 , a randomly chosen one is deleted .
permutations_. for randomly chosen @xmath181 , where @xmath182 is bounded from above by some convenient number to avoid too lengthy calculations , the segment of consecutive elements @xmath183 is randomly permuted .
it is evident that these five operations yield new sequences within the permitted set @xmath160 [ cf .
( [ x2 ] ) ] . paired insertions and paired deletions
are each others inverse operations , and the same holds for single insertions and single deletions .
these four types of operation together are sufficient for all elements of @xmath160 to communicate .
permutations are added to achieve better mixing properties .
the  detailed specification of how these elements are combined in the proposal distribution can be obtained from the authors .
two considerations guided this specification . in the first place , transparency of the algorithm . paired insertions and paired deletions are not always unique inverses of each other .
nonuniqueness of the inverse operation would lead to complicated counting procedures to determine proposal probabilities .
therefore , the restriction is made that pairs of elements @xmath86 can only be inserted at , or deleted from a pair of positions in the sequence , if there are no occurrences of the same @xmath86 between these positions ; and only if at least one other @xmath184 ( with @xmath185 or @xmath186 ) occurs in between these positions . in this restricted set of operations , paired
insertions and paired deletions are each other s unique inverses , which simplifies proposal probabilities . due to the presence of permutations , all elements of the space @xmath160 still are reachable from any element .
the  second consideration is computational efficiency . when proposing that a pair of elements @xmath86 is inserted or deleted at certain positions , then also proposing to permute a segment between those positions entails no increase in computational load for calculating the metropolis
hastings ratios , and this permutation will lead to larger changes in @xmath166 , and thereby , hopefully , better mixing for a given number of computations .      this subsection combines the elements presented in the preceding subsections to define an algorithm for mcmc approximation of the ml estimate .
it is now assumed that an arbitrary number @xmath14 of repeated observations has been made : @xmath187 .
as argued before , we condition on the first observation @xmath64 .
the  further data is denoted by @xmath188 .
the  parameter @xmath189 is denoted by @xmath126 .
the  markov property entails that the observed data score function , conditional on @xmath190 , can be decomposed as @xmath191\\[-8pt ] & & \qquad   =    \sum_{m=2}^m
s_{x(t_m ) \vert   x(t_{m-1})}(\theta ; x(t_m ) \vert   x(t_{m-1 } ) )   , \nonumber\end{aligned}\ ] ] where @xmath192 is the score function based on the conditional distribution of @xmath193 , given @xmath57 . for the period from @xmath194 to @xmath60 , the data set is augmented by @xmath195 which defines the order in which ties are changed between these time points , as described in section  [ s_aug ] ; this can be denoted by @xmath196 with outcome @xmath197 .
the  augmenting variable as used in section  [ s_aug ] is @xmath198 .
denote the probability ( [ full_like1 ] ) for the period from @xmath194 to @xmath60 by @xmath199 , and the corresponding total data score function by @xmath200 from ( [ mi1 ] ) and ( [ score_dec ] ) , and using the markov property , it can be concluded that the observed data score function now can be written as @xmath201\\[-8pt ] & & \qquad = \sum_{m=2}^m   e_\theta\ { s_m(\theta ; x(t_{m-1 } ) , v_m ) \vert   x(t_{m-1 } ) = x(t_{m-1 } ) , x(t_m ) = x(t_m ) \ }   .
\nonumber\hspace*{-20pt}\end{aligned}\ ] ] the  ml estimate is obtained as the value of @xmath126 for which ( [ mle4 ] ) equals 0 [ cf .
( [ mle1 ] ) ] .
the  algorithm is iterative , and the @xmath202th updating step now can be represented as follows :    1 .   for each @xmath66 ,
make a large number of the metropolis
hastings steps as described in section  [ s_sim ] , yielding @xmath203 .
2 .   compute @xmath204 using ( [ mle3 ] ) with @xmath205 defined by ( [ full_like1 ] ) for the period from @xmath194 to @xmath60 , and using ( [ kappa1 ] ) and ( [ kappa4 ] ) or ( [ kappa3 ] ) .
3 .   update the provisional parameter estimate by @xmath206    as mentioned in section [ s_appr ] , the estimate @xmath207 is calculated as a tail average of the values @xmath143 generated by this algorithm .
the  covariance matrix of the ml estimator is estimated using ( [ fi2 ] ) , where the expected values in the right - hand side are approximated by monte carlo simulation of the conditional distribution of  @xmath140 given @xmath208 , for @xmath209 .
this involves the matrix of partial derivatives ( [ fi1 ] ) , which can be estimated by a score - function method as elaborated in schweinberger and snijders ( @xcite ) .    the  main implementation details are the following :    1 .
the  method of moments ( mom ) estimator [ snijders ( @xcite ) ] , in practice , yields a good initial value @xmath210 .
2 .   to avoid long burn - in times for step ( 1 . ) , the metropolis
hastings algorithm for generating @xmath142 can be started from the previous value , @xmath211 , rather than independently .
this leads to some autocorrelation in the updates defined in step  ( 3 . ) , and the number of metropolis
hastings steps must be large enough that this autocorrelation is not too high .
it may be noted that the robbins monro algorithm is robust to moderate dependence between successive updates [ kushner and yin ( @xcite ) ] .
we have found good results when the number of steps is tuned so that the autocorrelations between the elements of @xmath212 are less than  0.3 . 3 .   for @xmath145 in step ( 3 . )
we use a monte carlo estimate of the complete data observed fisher information matrix ( [ fi1 ] ) , estimated for @xmath213 before making the iteration steps .
4 .   for the other numerical parameters of the algorithm we use the same values as described , for the stochastic approximation algorithm to compute the mom estimator , in snijders ( @xcite ) .
this algorithm is implemented in siena version 3.3 [ snijders et al .
( @xcite ) ]
. the  executable program as well as the code can be found athttp://www.stats.ox.ac.uk / siena/.
as an illustrative example , the data set is used that was also analyzed in van de bunt , van duijn and snijders ( @xcite ) and in snijders ( @xcite ) .
this is a friendship network between 32 freshman students in a given discipline at a dutch university .
initially most of the students were unknown to each other . there were six waves denoted @xmath214 of data collection , with 3 weeks between waves for the start of the academic year , and 6 weeks in between later .
the  relation studied is `` being friends or close friends . ''
the  data set is obtainable from website http://www.stats.ox.ac.uk/siena/[http://www.stats.ox.ac.uk/ ] http://www.stats.ox.ac.uk/siena/[siena/ ] .
the  transitions between observations @xmath73 to @xmath215 , and @xmath215 to @xmath216 , will here be studied separately . to identify the rate function ,
we assume ( arbitrarily but conveniently ) that the duration of the time periods is unity , @xmath217 . to keep the model specification relatively simple ,
the rate function is supposed to be constant across actors , given by @xmath105 from @xmath73 to @xmath215 and by @xmath218 from @xmath215 to @xmath216 ; and the objective function  ( [ fi ] ) is chosen independent of the previous state @xmath32 , containing contributions of the effects of outdegree , number of reciprocated ties , number of transitive triplets , number of 3-cycles , gender of the sender of the tie ( `` ego '' ) , gender of the receiver of the tie ( `` alter '' ) , and gender similarity : @xmath219 where variable @xmath220 indicates the gender of actor @xmath8 ( @xmath221 , @xmath222 ) and @xmath223 its average over the 32 individuals .
this model illustrates two types of triadic dependence ( transitive triplets and 3-cycles ) and the use of covariates ( gender ) .    for this model
the parameters are estimated for the two transitions @xmath224 and @xmath225 separately both by the method of moments ( mom ) and by maximum likelihood . for models where the functions @xmath226 used in ( [ fi ] )
depend only on @xmath5 and not on  @xmath32 , so that they can be expressed as @xmath227 , the mom estimator as defined by snijders ( @xcite ) is based on the vector with components @xmath228 for @xmath66 and @xmath229 for @xmath230 .
the  mom estimator is defined as the parameter vector for which the observed and expected values of this statistic are equal , and can be determined by stochastic approximation .
both the moment equation and the likelihood equation can be represented as @xmath231 , where @xmath232 is the difference between observed and estimated moments or the complete - data score function , respectively . for both estimators
, after running the stochastic approximation algorithm , convergence was checked by simulating the model for the obtained parameters for 2000 runs and calculating , for each component @xmath233 of @xmath232 , the ratio of the average simulated @xmath233 to its standard deviation . in all cases ,
this ratio was less than 0.1 , indicating adequate convergence .
@lld2.2cd2.2cd2.2cd2.2c@ & * effect * & & + & & & + & & & & & + & & & & & + & & & & & & & & & + & + @xmath23 & rate parameter & 3.95 & 0.64 & 2.61 & 0.36 & 3.43 & 0.59 & 5.67 & 0.75 + & + @xmath234 & outdegree & -1.66 & 0.26 & -1.02 & 0.26 & -2.19 & 0.30 & -2.00 & 0.22 + @xmath235 & reciprocated ties & 2.06 & 0.47 & 1.94 & 0.39 & 2.26 & 0.55 & 1.70 & 0.35 + @xmath236 & transitive triplets & 0.30 & 0.06 & 0.18 & 0.06 & 0.36 & 0.07 & 0.26 & 0.05 + @xmath237 & 3-cycles & -0.59 & 0.23 & -0.42 & 0.17 & -0.59 & 0.27 & -0.31 & 0.14 + @xmath238 & gender alter & 0.28 & 0.30 & 0.14 & 0.36 & 0.70 & 0.39 & 0.61 & 0.33 + @xmath239 & gender ego & -0.34 & 0.32 & -0.41 & 0.41 & -0.04 & 0.38 & -0.10 & 0.33 + @xmath240 & gender similarity & 0.33 & 0.30 & 0.34 & 0.36 & 0.48 & 0.37 & 0.44 & 0.32 +    parameter estimates and standard errors are reported in table [ tab1 ] . assuming that the estimators are approximately normally distributed ( which is supported by the simulations reported in the next section , although we have no proof ) , the significance can be tested by referring the ratios of estimate to standard error to a standard normal distribution .
the  method of moments and maximum likelihood estimates are different but lead to the same substantive conclusions .
the  parameters reflecting network structure , @xmath234 to @xmath237 , give the same picture for both transitions .
the  negative @xmath241 indicates that an outgoing tie which is not reciprocated and not transitively embedded is not considered attractive ; the positive @xmath242 and @xmath243 indicate that there is evidence for tendencies toward reciprocation and transitivity of friendship choices , when controlling for all other effects in the model . for interpreting the 3-cycles effect , note that closed 3-cycles are structures denying a hierarchically ordered relation .
the  negative @xmath244 indicates a tendency away from closed 3-cycles , which  in conjunction with the positive transitive triplets parameter  could be interpreted as a tendency toward a local ( i.e. , triadic ) popularity hierarchy in friendship . for gender
, there is only a close to significant effect of gender alter for the @xmath215@xmath216 transition , suggesting that male students tend to be more popular as friends , when controlling for the other effects .    from this single data example
, of course no conclusions can be drawn concerning the relative value of these two estimation methods .
a comparison between the finite sample behavior of the mom and the ml estimators can be based on simulations . here
we present a small simulation study as a very limited exploration of the relative efficiency of the two estimators , which is expected to be in favor of the ml estimator .
the  limited nature of this simulation study does not allow generalization , but the study design is meant to be typical for applications to friendship networks in rather small groups , and replicates approximately the empirical study of the previous section .
the  model is identical to that of the previous section : there are three repeated observations , 32 actors , one binary covariate called gender , and the first observed network as well as the distribution of the covariate are identical to the van de bunt data set at observation @xmath73 . therefore , the observation moments are again referred to as @xmath245 .
the  parameter values are rounded figures close to the estimates obtained in the preceding section .
networks for times @xmath215 and @xmath216 are generated and parameters estimated under the assumption that parameters @xmath246 are the same in periods @xmath247 and @xmath248 .
the  simulation model has parameters @xmath249 and @xmath250 , @xmath251 , and @xmath252 .
a total of 1000 data sets were generated and the estimates calculated by both methods .
table  [ tab2 ] reports the average estimates , the root mean squared errors , the rejection rates for testing the data - generating value of the parameter as the null hypothesis ( estimating type - i error rates ) , and the rejection rates for testing that the parameters equal 0 ( estimating power ) , where the tests were two - sided tests based on the @xmath24-ratio for the corresponding parameter estimate , assuming a standard normal reference distribution , at a nominal significance level of 5% .
@lld2.3d1.3ccd2.3d1.3cc@ & * effect * & & + & & & + & & & & & & & & & + & + @xmath249 & rate @xmath224 & 2.46 & 0.44 & 0.063 &  & 2.37 & 0.44 & 0.114 &  + @xmath250 & rate @xmath225 & 3.47 & 0.55 & 0.045 &  & 3.39 & 0.54 & 0.099 &  + & + @xmath253 & outdegree & -2.01 & 0.15 & 0.053 & 1.00 & -1.96 & 0.14 & 0.047 & 1.00 + @xmath254 & reciprocation & 1.03 & 0.26 & 0.044 & 0.96 & 0.97 & 0.26 & 0.042 & 0.95 + @xmath255 & transitivity & 0.186 & 0.052 & 0.043 & 0.94 & 0.180 & 0.054 & 0.064 & 0.93 + @xmath256 & 3-cycles & 0.02 & 0.14 & 0.042& & 0.03 & 0.14 & 0.060 &
+ @xmath257 & gender alter & 0.53 & 0.24 & 0.043 & 0.65 & 0.51 & 0.25 & 0.046 & 0.57 + @xmath258 & gender ego & -0.28 & 0.26 & 0.062 & 0.21 & -0.28 & 0.27 & 0.059 & 0.20 + @xmath259 & gender sim . & 0.52 & 0.24 & 0.031 & 0.64 & 0.52 & 0.25 & 0.052 & 0.60 +    out of the 1000 generated data sets , 5 were excluded because they did not produce well converging results in the default specification of the algorithm for one or both estimators .
table [ tab2 ] shows that the results for the two estimators are very similar , and the type - i error rates are close to the nominal value , except for the inflated type - i rates of the ml estimators for the two rate parameters .
the  latter is related to the skewed distribution of the rate parameter estimators .
the  correlations between the estimators are more than 0.93 for all coordinates ; note that correlations are attenuated due to the stochastic nature of the algorithms .
it can be concluded that for this type of model , characterized by 32 actors and 3 waves with rates of change 2.5 and 3.5 , with 7 parameters in the objective function , mom and ml estimation yield quite similar results .    to explore data sets with less information ,
a similar simulation study was conducted with 20 actors , where the first network was induced by the @xmath73 network of  20 of the actors in the van de bunt data set , and the rest of the simulation design differed from the previous study by including an interaction effect of reciprocity by gender similarity , represented by the term @xmath260 with parameter @xmath261 .
this effect is included to achieve a higher correlation of the parameter estimators , which together with the smaller number of actors should lead to greater difficulties in estimation .
@lld2.3d1.3ccd2.3d1.3cc@ & * effect * & & + & & & + & & & & & & & & & + & + @xmath249 & rate @xmath224 & 2.49 & 0.50 & 0.059 &  & 2.37 & 0.48 & 0.093 &  + @xmath250 & rate @xmath225 & 3.51 & 0.70 & 0.044 &  & 3.36 & 0.67 & 0.094 &  + & + @xmath253 & outdegree & -2.09 & 0.29 & 0.009 & 0.98 & -2.02 & 0.21 & 0.025 & 1.00 + @xmath254 & reciprocation & 1.08 & 0.35 & 0.043 & 0.86 & 0.97 & 0.33 & 0.052 & 0.82 + @xmath255 & transitivity & 0.170 & 0.099 & 0.022 & 0.51 & 0.158 & 0.096 & 0.059 & 0.52 + @xmath256 & 3-cycles & 0.01 & 0.23 & 0.034& & 0.05 & 0.23 & 0.057 &
+ @xmath257 & gender alter & 0.59 & 0.44 & 0.020 & 0.15 & 0.60 & 0.38 & 0.048 & 0.43 + @xmath258 & gender ego & -0.35 & 0.50 & 0.024 & 0.00 & -0.34 & 0.36 & 0.034 & 0.11 + @xmath259 & gender sim . &
0.61 & 0.50 & 0.013 & 0.12 & 0.63 & 0.43 & 0.051 & 0.35 + @xmath262 & g. sim .
@xmath263 rec . &
0.47 & 0.63 & 0.039 & 0.10 & 0.50 & 0.60 & 0.043 & 0.14 +    the  results are shown in table [ tab3 ] .
of the 1000 generated data sets , 20 were excluded from the results because of questionable convergence of the algorithm .
the  table shows that the ml estimator here performs clearly better than the mom estimator .
the  estimated relative efficiency of mom compared to ml expressed as ratio of mean squared errors ranges for these 10 parameters from 0.50 to 0.99 .
the  correlation between the estimators ranges from 0.71 ( gender ego ) to 0.98 ( rate period 1 ) .
for all parameters except @xmath235 , the estimated power of the test based on the ml estimator is higher than that of the mom - based test , which can be traced back to the combination of a smaller mean squared error and a less conservative test .
the  surprisingly low power of the mom - based test for the gender - ego effect reflects high standard errors that tend to be obtained for estimating @xmath239 .
one of the important advantages of a likelihood approach is the possibility of elaborating model selection procedures .
here we only explain how to conduct a likelihood ratio test .    a convenient way to estimate the likelihood ratio is by a simple implementation of the idea of the path sampling method described by gelman and meng ( @xcite ) , who took this method from the statistical physics literature where it is known as thermodynamical integration . for arbitrary @xmath264 and @xmath265 , defining the function @xmath266 , with @xmath267 , this method is based on the equation @xmath268 this integral can be approximated by replacing the integral by a finite sum and using ( [ mi1 ] ) . applying a simulation procedure which for each @xmath269 , @xmath270 , generates @xmath271 draws @xmath272 from the conditional distribution of @xmath140 given @xmath208 under parameter @xmath269 , the log likelihood ratio ( [ llr ] )
can be approximated by @xmath273 burn - in time can be considerably reduced when starting the mcmc algorithm for generating @xmath274 by the end result of the algorithm used to generate @xmath275 .
this was applied to the van de bunt data set used in section  [ sec4 ] to test the null hypothesis that all parameters @xmath276 are for period @xmath248 the same as for @xmath247 against the alternative that they are allowed to be different .
the  rate parameters @xmath23 were allowed to be different under the null as well as the alternative hypothesis . for calculating ( [ path_samp ] ) we used @xmath277 , @xmath278 .
the  estimated likelihood ratio was 18.7 , which for a @xmath279 distribution yields @xmath280 , leading to rejecting the null hypothesis at conventional levels of significance .
this article has presented a model for longitudinal network data collected in a panel design and an algorithm for calculating the ml estimator .
the  model can represent triadic and other complicated dependencies that are characteristic for social networks .
it is designed for applications in social network analysis , but related models may be applied for modeling networks in other sciences , such as biology .
earlier , a method of moments ( mom ) estimator for this model was proposed by snijders ( @xcite ) and bayesian inference methods by koskinen and snijders ( @xcite ) .
the  algorithm was constructed using stochastic approximation according to the approach proposed by gu and kong ( @xcite ) , and employs monte carlo simulations of the unobserved changes between the panel waves , conditional on the observed data .
the  simulation design used is more efficient than that used in koskinen and snijders ( @xcite ) because here the waiting times between unobserved changes are integrated out .
no proof is yet available for the consistency and asymptotic normality of the ml estimator , which are intuitively plausible for the situation where @xmath281 tends to infinity , @xmath72 and @xmath73 are fixed , and the parameters are such that the average degree @xmath282 tends to a positive finite limit .
limited simulation results do support the expectation that the estimators are asymptotically normal .
the  nonstandard assumptions ( lack of independence ) imply , however , that a proof may be expected to be rather complicated .
our experience in the reported simulations and in working with empirical data sets is that the algorithm converges well unless data sets are small given the number of estimated parameters , but the algorithm is time - consuming ( e.g. , each ml estimation for one data set in table [ tab2 ] took about 35 minutes on a regular personal computer , while each mom estimation took about 2 minutes ) .
further improvements in the algorithm are desirable for the practical use of ml estimators in these models .
this is the subject of future work .
judging from our very limited simulations , the advantages of ml estimation over mom estimation in terms of root mean squared error and power of the associated tests seem strong for small data sets and small for medium to large data sets .
further simulation studies are necessary .
however , even if the statistical efficiency is similar , likelihood - based estimation can have additional advantages , for example , the possibility of extensions to more complicated models and of elaborating model selection procedures .
software implementing the procedures in this paper is available athttp://www.stats.ox.ac.uk / siena/.
part of this work was done while m. schweinberger was working at the university of groningen and the university of washington .
part of this work was done while j. koskinen was working at the university of stockholm and the university of melbourne .
99 airoldi , e. , blei , d. m. , fienberg , s. e. , goldenberg , a. , xing , e. p. and zheng , a. x. ( 2007 ) .
_ statistical network analysis : models , issues and new directions ( icml 2006)_. _ lecture notes in computer science _ * 4503*. springer , berlin .
hanneke , s. and xing , e. p. ( 2007 ) .
discrete temporal models of social networks . in _ statistical network analysis : models , issues and new directions ( icml 2006 ) _ ( e. airoldi , d. m. blei , s.  e.  fienberg , a. goldenberg , e. p. xing and a. x. zheng , eds . ) . _ lecture notes in computer science _ * 4503 * 115125 .
springer , berlin .
krackhardt , d. and handcock , m. s. ( 2007 ) .
heider vs simmel : emergent features in dynamic structures . in _
statistical network analysis : models , issues and new directions ( icml 2006)_. ( e. airoldi , d. m. blei , s.  e.  fienberg , a. goldenberg , e. p. xing and a. x. zheng , eds . ) _ lecture notes in computer science _
* 4503 * 1427 .
springer , berlin .
orchard , t. and woodbury , m. a. ( 1972 ) .
a missing information principle : theory and applications . in _
proceedings sixth berkeley sympos . math .
probab . _ * 1 * 697715
california press , berkeley .
snijders , t. a. b. and van duijn , m. a. j. ( 1997 ) .
simulation for statistical inference in dynamic network models . in
_ simulating social phenomena _ ( r. conte , r. hegselmann and p. terna , eds .
) 493512 .
springer , berlin .
van de bunt , g. g. , van duijn , m. a. j. and snijders , t. a. b. ( 1999 ) .
friendship networks through time : an actor - oriented statistical network model . _
computational and mathematical organization theory _ * 5 * 167192 .
van duijn , m. a. j. , zeggelink , e. p. h. , huisman , m. , stokman , f. m. and wasseur ,  f.  w. ( 2003 ) .
evolution of sociology freshmen into a friendship network .
_ journal of mathematical sociology _ * 27 * 153191 .
wasserman , s. ( 1979 ) . a stochastic model for directed graphs with transition rates determined by reciprocity .
_ sociological methodology1980 _ ( k. f. schuessler , ed . ) 392412 .
jossey - bass , san francisco .