modelling and forecasting electricity loads is a problem well - known within both the academic and the applied statistics community ( see e.g. * ? ? ?
the signals studied usually exhibit strong properties such as seasonalities or weekly and daily profiles , leading to some very accurate models that tend to perform rather well under normal forecasting conditions .
the approaches used to model and forecast them vary a lot : we mention a couple of them in the lines below .
some authors worked with univariate time series models : @xcite built a double seasonal exponential smoothing for the british electricity load , while @xcite and @xcite presented some comparative studies between univariate methods for different sets of data .
@xcite opted for a non - parametric approach relying on the wavelet transform to forecast the load curve seen as a functional - valued autoregressive hilbertian process .
others tried and modelled the load together with the use of exogenous variables : @xcite included the temperature in their model , which inspired the bayesian semi - parametric regression model found in @xcite .
alternatives to univariate modelling were often considered too , such as building multiple - equation models : while the various hours of the day share the same equation , the associated parameters differ from one another .
@xcite built an hourly independent seasonal auto - regressive model for their data , and @xcite also built an independent model for each hour of the day but took temperature effects into account .
a state - space model is proposed in @xcite and @xcite where the parameters of the model are also allowed to vary along the time .
the exogenous variables most commonly used to forecast the electricity load are weather - based , even though the decision to include one or more meteorological variables into a model may be open to discussion .
the period of forecasting has to be taken into account , as well as the accuracy of the predictions for such variables . for temperate climates
, the most important meteorological factor is the temperature ( see e.g. * ? ? ?
for the french electricity load specifically , the importance of the temperature and cloud - cover was underlined in @xcite and @xcite .
other weather - related models include the works of @xcite and @xcite within the bayesian framework .
let us also mention machine learning work : @xcite for a neural networks implementation and @xcite for a detailed study of online mixing algorithms used on a set of various predictors .
we are interested in the development of a methodology to improve the estimation and the predictions of a parametric multi - equation model ( similar to the one presented in @xcite ) over a short dataset .
the limited size of the dataset coupled with the high dimensionality of the model leads to a typical overfitting situation when using the maximum likelihood approach : the fitted values are relatively close to the observations while the errors in prediction are an order of magnitude larger or more ( note that due to the very periodic nature of its regressors , the model typically requires 4 or 5 years of data to provide satisfactory predictions ) .
this overfitting behaviour can be somewhat alleviated by the use of a bayesian estimation relying on an informative prior distribution , but the very fact that the data available is limited makes the posterior distribution all the more sensitive to the choice of that prior .
although electricity load curves may largely differ from one population to another , they may also share some common features .
the latter case is expected to happen when the global population studied is an aggregation of non - homogeneous subpopulations for which the estimations are made harder due to the relative lack of data .    to design a sensible prior in such a situation
, we consider the case where another long dataset is available , upon which the model performs equally well in both estimation and prediction .
we assume the long and the short datasets are somehow similar in a non - obvious way . that the similarity between the parameters underlying the two datasets ( we will assume they are indeed coming from the model considered ) can not be easily guessed prevents us from trying to model the datasets simultaneously because it would require a rather precise knowledge of the link between the two .
we propose a general way of building an informative hierarchical ( see * ? ? ?
* for a general review on the subject of hierarchical models ) prior for the short dataset from the long one that goes as follows :
1 .   we first estimate the posterior distribution on the long dataset using a non - informative prior , arguing that the design of an informative prior for this dataset is not necessary , since the data available is enough to estimate and predict the model in this case ; 2 .   we extract key pieces of information from this estimation ( e.g. moments ) to design an informative prior for the short dataset which takes into account the prior information that the datasets are somehow similar , via the introduction of hyperparameters designed to model and estimate this similarity .
the paper is organised as follows . in section [ sec : methodo ] we focus on the general methodology and describe the way we carried our experimentation , we also present the general regression model used for our tests and applications . in section [ sec : specpriors ] , we present the semi - conjugated priors ( informative and non - informative ) used on each of the datasets .
the ad hoc mcmc algorithms we developed to estimate the mean and variance of the posterior distributions are push backed into the appendix so as not to obfuscate the main point of the paper by technical details . in section
[ sec : simulations ] , we use these algorithms to illustrate and validate our approach in simulated situations : we show the contribution of the informative prior over the precision of both the estimated parameters and the forecasts in the case of a working electricity load forecasting model . in section [ sec : applications ] , we apply our method to french electricity datasets and compare the results with the outputs of 3 alternative standard methods to assess its competitivity .
we also study the effect of the small dataset s sample size upon the predictive quality of the model and show that the informative prior provides reasonable forecasts even when the lack of data is severe .
let us define here some notations that we shall keep throughout this paper .
hereafter , we denote @xmath0 a short dataset over which we would like to estimate the model and we denote @xmath1 a long dataset known or thought to share some common features with @xmath0 .
we will denote @xmath2 the parameters of the model and @xmath3 the observations from @xmath1 .
we propose a method designed to help improve parameter estimations and model predictions over @xmath0 with the help of @xmath1 .
let @xmath4 be the prior distribution used on @xmath1 and @xmath5 the associated posterior distribution .
observe that the choice of @xmath4 is not crucial as long as it remains non - informative enough because the model can be correctly estimated from the data alone on @xmath1 .
let @xmath6 denote the prior distribution to be used on @xmath0 .
notice that the naive pick @xmath7 is not a viable solution as soon as the parameters of @xmath1 and @xmath0 differ since the variance of the posterior distribution @xmath5 is too small : in that case the data of @xmath0 will not be able to make up for the difference between the posterior mean on @xmath1 and the true value of the parameters of @xmath0 .
assuming that the parameters corresponding to @xmath1 and @xmath0 are identical is too restrictive in practise . to allow for more flexibility
we add hyperparameters accounting for the similarity between the datasets .
we now described the informative hierarchical prior we designed .
assume that the prior distribution @xmath6 is to be chosen within the parametric family @xmath8 since selecting @xmath9 is equivalent to picking @xmath10 , and since we want @xmath11 to retain some key - features of @xmath5 , we want to pick @xmath12 using some of the information contained inside the posterior distribution obtained on @xmath1 .
we assume that there exists an operator @xmath13 , such that @xmath14 = \lambda,\ ] ] and choose @xmath12 proportional to @xmath15 $ ] , in the sense that @xmath16,\ ] ] where @xmath17 itself is an unknown linear operator that we assume diagonal for ease of use .
the operator @xmath18 can be interpreted as a similarity operator between @xmath1 and @xmath0 , and its diagonal components as similarity coefficients measuring how close the two datasets really are when looked at through @xmath19 .
the diagonal components of @xmath18 are hyperparameters of the prior we designed , and we give them a vague hierarchical prior distribution centred around @xmath20 , the prior on @xmath20 being vague and centred around @xmath21 .
the hyperparameter @xmath20 may also be regarded as a more global similarity coefficient , since it represents the mean of all the similarity coefficients .
the prior mean of @xmath20 is forced to @xmath21 to reflect the prior knowledge that the datasets are somehow similar .
the variance of the prior distribution of @xmath20 could in theory be reduced , going from a vague prior to a more informative structure , depending on the confidence we have over the similarity between the datasets .
we chose not to however , so as to keep the procedure we describe from requiring any delicate subjective adjustments .
we present now two frequent situations where the above procedure can be written in a simpler way .
we assume that the elements of @xmath22 can be identified via their @xmath23 first moments : the operator @xmath19 can then be reduced to a function @xmath24 of the @xmath23 first moments operators , i.e. @xmath25 = f({\mathbb{e}}(\theta ) , \ldots , { \mathbb{e}}(\theta^m))$ ] .
the expression of @xmath12 then becomes @xmath26 note that , if the prior requires the specification of at least the two first moments , even though the priors from the upper layers of the model are vague , the correlation matrix estimated on the dataset @xmath1 remains untouched and is directly plugged into in the informative prior if we consider centred moments for orders greater than @xmath21 .
we consider the case where @xmath22 is the family of priors conjugated for the model .
if the prior @xmath4 belongs to @xmath22 then the associated distribution @xmath5 does too and there corresponds a parameter @xmath27 to it .
the expression of @xmath12 thus reduces to @xmath28      modelling and forecasting the electricity load ( or demand ) on a day - to - day basis has long been a key activity for any company involved in the electricity industry .
it is first and foremost needed to supply a fixed voltage at all ends of an electricity grid : to be able to do so , the amount of electricity produced has to match the demand very closely at any given time and experts usually make use of short - term forecasts with this aim in view as mentioned in @xcite .
electricity load usually has a large predictable component due to its very strong daily , weekly and yearly periodic behaviour .
it has also been noted in many regions that the weather usually affects the load too , the most important meteorological factor typically being the temperature ( see * ? ? ? * for an example ) .    for each of the 48 instants of the day ( each instant lasts 30 minutes , starting from 00:00am ) , the non - linear regression model that we consider in this paper , first described in @xcite ,
is made of three components , which we explain briefly in the next paragraphs , and is usually formulated as follows : for @xmath29 , @xmath30 + \sum_{j=1}^{d_{12 } } \omega_j { \mathds{1}}_{\omega_j}(t ) ,                          \nonumber\\ x^{(2)}_t & = \sum_{j=1}^{d_2 } \psi_j { \mathds{1}}_{\psi_j}(t ) ,                    \nonumber\\ x^{(3)}_t & = g ( t_t - u){\mathds{1}}_{{[t_t,\,+\infty[}}(u ) ,                 \nonumber\end{aligned}\ ] ] where @xmath31 is the load of day @xmath32 and where @xmath33 are assumed independent and identically distributed with common distribution @xmath34 .
the @xmath35 component is meant to account for the average seasonal behaviour of the electricity load , with a truncated fourier series ( whose coefficients are @xmath36 and @xmath37 ) and gaps ( parameters @xmath38 ) which represent the average levels of electricity load over predetermined periods given by a partition @xmath39 of the calendar .
this partition usually specifies holidays , or the period of time when daylight saving time is in effect i.e. major breaks in the electricity consumption behaviour . the left part of figure [ fig : illuconso ] shows a typical behaviour over two different periods of time ( summer vs. winter ) .
the @xmath40 component allows for day - to - day adjustments of the seasonal behaviour @xmath35 through shapes ( parameters @xmath41 ) that depends on the so - called days types which are given by a second partition @xmath42 of the calendar .
this partition usually separates weekdays from weekends , and bank holidays .
the differences between two different daytypes are visible on the left part of figure [ fig : illuconso ] too . for obvious identifiability reasons ,
the vector @xmath43 is restricted to the positive quadrant of the @xmath44-unit sphere in @xmath45 , that we denote @xmath46    the @xmath47 component represents the non - linear heating effect that links the electricity load to the temperature ( see * ? ? ? * for a general presentation of non - linear models ) , with the help of 2 parameters .
the heating threshold @xmath48}$ ] corresponds to the temperature above which the heating effect is considered null and is usually estimated to be roughly around 15c .
the heating effect is supposed to be linear for temperatures below the threshold and null for temperatures above .
the restriction on the support of the threshold @xmath49 simply expresses the fact that the threshold is sought within the range of the observed temperatures , i.e. @xmath48}$ ] with @xmath50 the heating gradient @xmath51 where @xmath52 represents the intensity of the heating effect , i.e. the slope ( assumed to be non - zero ) of the linear part that can be observed on the right part of figure [ fig : illuconso ] .    using the notation @xmath53 for the @xmath54-th row of a matrix @xmath55
, the previous model can be re - written in the following condensed and more generic way : for @xmath56 @xmath57 the matrices @xmath58 of size @xmath59 , @xmath60 of size @xmath61 , @xmath62 of size @xmath63 , and @xmath19 of size @xmath63 are known exogenous variables while the parameters of the model to be estimated are @xmath64 } \times { \mathbb{r}}_+^*,\ ] ] where @xmath65 is the positive quadrant of the @xmath44-unit ball of dimension @xmath66 .
one could , without too much difficulty , add a cooling effect to the model , whose definition would be similar to that of the heating effect . since the cooling effect remains far less important than the heating effect in france at the present time ( see the right part of figure [ fig : illuconso ] ) , and since the estimation of the associated cooling threshold is often unstable at best , we felt that adding such a part to the model was not as crucial as it would be in other countries where the cooling effect plays a much more important role .
for the applications presented in section [ sec : applications ] of this paper , we thus went for a simpler implementation : given a cooling threshold @xmath67 , a regressor whose coordinates are @xmath68-\infty,\,t_t]}}(u^c)$ ] , for @xmath69 is added to the matrix @xmath58 .
it models , in practise , a cooling effect with a fixed cooling threshold and an estimated cooling gradient that is multiplied by the daytype effect . considering the expression in
, the model is quite general since the bulk of it could be thought of as the product of two linear regressions , with the added twist of a non - linearity introduced via the threshold parameter @xmath49 ( change - point of the model ) .
even though the priors and algorithms constructed in the coming sections do depend on the model introduced here , they can be modify in a rather straightforward manner , should the reader want to tweak the model a bit ( for example to add another exogenous variable such as the wind or the cloud - cover ) .
hereafter @xmath70 will denote the likelihood of the observations @xmath71 .
we will often write the model for @xmath72 as @xmath73 and use the notation @xmath74 for short , where @xmath75 designate the parameters of interest . with these notations , since @xmath76 , the likelihood of the model described in reads : @xmath77
let us denote @xmath78 and @xmath79 the posterior mean and posterior variance of @xmath80 from a non - informative approach applied to the long dataset @xmath1 , that we assume have already been collected . for the sake of clarity
, we drop the @xmath81 notation : when not explicitly specified , the dataset and observations as well as the prior and posterior distributions we refer to in this section will be those corresponding to @xmath0 .
we now present the informative hierarchical prior for the model , and then prove that it leads to a proper posterior distribution ( see proposition [ hierarchicalproperpost ] ) . following the methodology exposed in section [ subsec : generalprinciple ] ,
the informative prior that we propose introduces new parameters to model the similarity between the two datasets , @xmath82 and @xmath83 such that @xmath84 where @xmath85 .
the coordinates of the vector @xmath86 can be interpreted as similarity coefficients between parameters of @xmath87 and @xmath88 and the strictly positive scalar @xmath89 can be seen as a way to alternatively weaken or strengthen the covariance matrix as needed .
hyperparameters @xmath20 and @xmath90 are more general indicators of how close @xmath1 and @xmath0 are , @xmath20 corresponding to the mean of the coordinates of @xmath86 and @xmath90 being their inverse - variance .
@xmath89 , @xmath20 , @xmath90 and @xmath91 of course require a prior distribution too . for @xmath91
we use a non - informative prior ( we chose @xmath92 ) because we do not want to make any kind of assumptions about the noise around both datasets .
this prior is non - informative in the sense that it matches jeffreys prior distribution on @xmath91 for a gaussian linear regression .
for the three other parameters , based on semi - conjugacy considerations , we use : @xmath93 where @xmath94 , @xmath95 and @xmath96 are fixed positive real numbers such that the prior distribution on @xmath89 , @xmath20 and @xmath90 are vague .
these prior distributions are chosen because of their conjugacy properties ( as will be seen in the mcmc algorithm ) .
the vagueness requirement that we impose on these priors is motivated by the fact that we want to keep as general a framework as possible without having to tweak each and every prior coefficient for different applications .    in the end ,
the informative prior is built as follows : @xmath97 recalling the notations introduced at the end of section [ sec : methodo ] , the posterior measure is given by @xmath98}\times{[\underline{u},\,\overline{u}]}\times { \mathbb{r}}_+^*}(\|\beta\|_1,u,\sigma^2 ) \nonumber \\ & \quad\quad\times l^{\frac{d}{2 } } \exp\left(-\frac{1}{2}(\eta - k\mu^{{\mathcal{a}}})^\prime l ( \sigma^{{\mathcal{a}}})^{-1}(\eta - k\mu^{{\mathcal{a}}})\right ) \nonumber \\ & \quad\quad\times |r|^{\frac{d}{2 } } \exp\left(-\frac{1}{2}r\sum_{i=1}^d ( k_i - q)^2\right ) l^{a_l-1 } \exp\left(-b_l l\right ) { \mathds{1}}_{{\mathbb{r}}_+^*}(l ) \nonumber \\ & \quad\quad\times |\sigma_q^{-2}|^{\frac{1}{2 } } \exp\left(-\frac{1}{2}\sigma_q^{-2 } ( q-1)^2\right ) r^{a_r-1 } \exp\left(-b_r r\right ) { \mathds{1}}_{{\mathbb{r}}_+^*}(r ) .
\nonumber\end{aligned}\ ] ]    [ hierarchicalproperpost ] for @xmath99}$ ] denote @xmath100 the matrix whose rows are @xmath101 , \quad t=1,\ldots , n,\ ] ] and suppose @xmath102 has full rank for every @xmath99}$ ] .
assume furthermore that @xmath103 and that @xmath104 are observations coming from the model and the posterior measure is then a well - defined ( proper ) probability distribution .
first notice that @xmath105 is proportional to @xmath106}}(\|\beta\|_1){\mathds{1}}_{{[\underline{u},\,\overline{u}]}}(u ) \pi(\eta|k , l)\pi(k | q , r)\pi(l)\pi(q)\pi(r),\end{aligned}\ ] ] for almost every @xmath107 and that the function @xmath108 is bounded , for almost every @xmath107 .
the posterior integrability is hence trivial as long as @xmath109 itself is a proper distribution which is the case here .
we now propose a non - informative prior to use with the long dataset @xmath1 . note that since the dataset @xmath1 is long enough
, the choice of the prior distribution used in this situation does not matter much as long as it remains vague enough : the bayes estimator is expected to converge , as the number of observations grows , to the maximum likelihood estimator of the model we use .
the main issue when studying the asymptotical behaviour of the posterior distribution or the maximum likelihood estimator is that the likelihood of the model is not continuously differentiable with regard to the heating threshold . isolating the heating part from the rest of the model , @xcite
show that this issue can be dealt with and proove among other results that both the bayes estimator and the maximum likelihood estimator are consistent .
the non - informative prior is thus to be considered hereafter as an equivalent to the maximum likelihood approach for all intents and purposes .    for the sake of clarity again
, we drop the @xmath110 notation : when not explicitly specified , the dataset and observations as well as the prior and posterior distributions we refer to in section will be those corresponding to @xmath1 .
we show that the use of a non - informative prior distribution leads to a proper posterior distribution ( see proposition [ noninfoproperpost ] ) .
we use the following non - informative prior @xmath111 this prior is non - informative in the sense that it matches jeffreys prior distribution on @xmath91 for a gaussian linear regression and matches laplace s flat prior on the other parameters .
it leads to the following posterior distribution @xmath112}\times{[\underline{u},\,\overline{u}]}\times { \mathbb{r}}_+^*}(\|\beta\|_1,u,\sigma^2 ) .
\nonumber\end{aligned}\ ] ]    [ noninfoproperpost ] for @xmath99}$ ] denote @xmath100 the matrix whose rows are @xmath101 , \quad t=1,\ldots , n,\ ] ] and suppose @xmath102 has full rank for every @xmath99}$ ] .
assume furthermore that @xmath103 and that @xmath104 are observations coming from the model , the posterior measure is then a well - defined ( proper ) probability distribution .
notice first that @xmath113}}(\|b\|_1){\mathds{1}}_{{[\underline{u},\,\overline{u}]}}(u ) \quad \text{for almost every } y,\end{aligned}\ ] ] and observe then that @xmath114 ^ 2.\end{aligned}\ ] ] let @xmath115}$ ] and denote @xmath116 .
we write @xmath117 ^ 2 \\ & = \|y - a_*(\beta_0 , u_0 ) \alpha_*\|_2 ^ 2,\end{aligned}\ ] ] and thus obtain the following equivalence , as @xmath118 and @xmath119 @xmath120 the triangular inequality applied to the right hand side of gives @xmath121 since @xmath122 has full rank , by straightforward algebra we get @xmath123 where @xmath124 is the smallest eigenvalue @xmath125 and is strictly positive .
we can hence find an equivalent of the right hand side of as @xmath119 , which is @xmath126 combining , and together , we see that the integrability of the left hand side of as @xmath118 and @xmath119 is directly implied by that of @xmath127 .
the latter is of course immediate for @xmath128 as can be seen via a quick cartesian to hyperspherical re - parametrisation .
the previous paragraph thus ensures the integrability of @xmath129 over sets of the form @xmath130m(\beta_0,u_0),\,+\infty[}\ } , \quad \forall ( \beta_0 , u_0 ) \in b_+^{d_{\beta}}(0 , 1 ) \times { [ \underline{u},\,\overline{u}]}\ ] ] where the subset @xmath131 is an open neighbourhood of @xmath132 and @xmath133 is a real number depending on @xmath132 . by compacity of @xmath134}$ ] there exists a finite union of such @xmath135 that covers @xmath134}$ ] . denoting @xmath55 the maximum of @xmath136 over the corresponding finite subset of @xmath137
, we finally obtain the integrability of @xmath129 over @xmath138m,\,+\infty[}\}$ ] .
the integrability of @xmath129 over @xmath139}\}$ ] is trivial , recalling that @xmath140 is continuous and does not vanish over this compact for almost every @xmath107 , meaning its inverse shares these same properties .
the condition `` @xmath141 has full rank '' mentioned above is typically verified in our applications for the regressors used in our model . to see this ,
call `` vector of heating degrees '' the vector whose coordinates are @xmath142 , then not verifying the aforementioned condition is equivalent to saying that there exists an index @xmath54 and a threshold @xmath49 such that the family of vectors formed by the regressors @xmath58 and the vector of heating degrees is linearly dependent over the subset @xmath143 of the calendar  .
in this section we simulate a long dataset @xmath1 and a short dataset @xmath0 from the model to assess the performance of the informative prior as the similarity between the datasets varies . to measure the improvement brought by the informative prior we compare the estimation and prediction on dataset @xmath0 with a non - informative prior .
for any estimation ( posterior mean and variance ) on a dataset ( be it @xmath1 or @xmath0 ) , the mcmc algorithms would typically run for 500,000 iterations after a small burn - in period .      _ predictive distribution . _
the bayesian framework allows us to compute so - called predictive distributions , i.e. the distributions of future observations given past observations .
given a prior distribution @xmath144 and the corresponding posterior distribution @xmath145 related to the past observations @xmath71 , the predictive distribution for the future observation @xmath146 is defined as @xmath147 \\ & = \int y_{n+k } g(y_{n+k}|y ) { \,\mathrm{d}}y_{n+k}. \label{optimalbayespred}\end{aligned}\ ] ]    _ the comparison criterion .
_ to assess the quality of the estimation of the model with our informative prior with regard to the estimation of the model with the non - informative prior , we compare both results based on the quality of the predictions .
let @xmath148 be the upcoming observation , the prediction error can be written as @xmath149 + [ f_{n+1}(\eta_0)-\widehat{y}_{n+1}],\ ] ] which expresses the prediction error as a sum of a noise @xmath150 ( whose theoretical distribution is @xmath151 ) and a bias which can be seen as an estimation error over the prediction @xmath152 .
we focus solely on the second part , since the first part ( the noise ) is unavoidable in real situation . given that we want to validate our model on simulated data ,
the quantity @xmath152 is indeed accessible here whereas it would not be in real situation .
we thus choose to consider the quadratic distance between the real and the predicted model over a year as our quality criterion for a model , i.e. : @xmath153 ^ 2}.\label{comparisoncriterion}\end{aligned}\ ] ]      both datasets @xmath1 and @xmath0 were simulated according to the model with @xmath154 ( 4 frequencies used for the truncated fourier series ) .
the calendars and the partitions used for @xmath1 and @xmath0 were designed to include 7 daytypes ( @xmath155 , one daytype for each day of the week ) , but did not include any special days such as bank holidays .
they also included 2 offsets ( @xmath156 ) to simulate the daylight saving time effect . in the end
we thus had @xmath157 and @xmath158 i.e. @xmath159 using the expression of the model given in .
_ dataset @xmath1 .
_ we simulated 4 years of daily data for @xmath1 with parameters : @xmath160 these values were chosen to approximately mimic the typical electricity load of france up to a scaling factor .
the temperatures we used for the estimation over @xmath1 are those measured from september 1996 to august 2000 at 10:00am .
_ dataset @xmath0 .
_ we simulated 1 year of daily data for @xmath0 with parameters : @xmath161 where the coordinates of the true hyperparameters @xmath86 were allowed to vary around @xmath21 .
the temperatures we used for the estimation over @xmath0 are those measured from september 2000 to august 2001 at 10:00am .
we also simulated an extra year of daily data @xmath0 for prediction , with the same parameters but using the so - called normal temperatures , meaning that for each day of this extra year the temperature is the mean of all the past temperatures at the same time of the year .
we made such a choice to try and suppress any dependency between our simulated results and the chosen temperature for this fictive year of prediction , since we did not want to bias our results because of a rigorous winter or an excessively hot summer .
we chose to use vague priors ( i.e. proper distributions with large variances ) for the uppermost layers of our informative hierarchical prior , and thus decided to use the values : @xmath162 a study of the bayesian hierarchical model s sensitivity to these values showed that changing these hyperparameters to achieve prior variances of greater magnitudes hardly influenced the posterior results ( means and variances ) at all .
this is why we decided to stick to these values for the remainder of our experimentations .    _
_ we benchmarked the bayesian model with its informative prior against its non - informative prior counterpart for different choices of true hyperparameters @xmath86 over 300 replications ( data being simulated anew for each replication ) , i.e. we simulated many different datasets @xmath0 looking more or less similar to @xmath1 and applied our method on them .
figure [ fig : mean10 ] shows the posterior error of @xmath80 ( posterior mean minus the true value ) of @xmath80 , based on 300 replications that correspond to the case where @xmath163 i.e @xmath164 for both the informative ( leftmost ) and non - informative ( rightmost ) method .
marginal confidence interval for the posterior means are much smaller when using the informative prior ( most of them hitting the true value ) .
the marginal posterior standard deviations ( not shown here ) are also reduced when the informative prior is used instead of the non - informative prior .
( seasonal parameters ) , @xmath165 ( shape parameters ) , and @xmath166 and @xmath49 ( heating parameters ) , based on 300 replications .
leftmost replications correspond to the informative method while the rightmost replications correspond to the non - informative method . here
@xmath163 . ]
except @xmath167 and @xmath168 . ]
( seasonal parameters ) , based on 300 replications .
top row is for the case were @xmath163 and bottom row is for the case where @xmath167 and @xmath168 .
leftmost replications correspond to the informative method while the rightmost replications correspond to the non - informative method .
posterior errors of @xmath169 ( shape parameters ) , and @xmath170 and @xmath171 ( heating parameters ) are not shown here because no significant deviation from @xmath172 was found on either of these coordinates when the informative prior was used in either case ( the empirical variances on these coordinates were bigger in the non - ideal case though , in a similar fashion to what we observe here for @xmath173 ) . ]
when the situation is far from being as ideal as the one mentioned above , the informative approach still shows improvement over the non - informative approach but to a lesser extent .
figure [ fig : seasonheat_mean05 ] shows that the estimations of some of the parameters of the model are improved with the addition of the prior information ( @xmath174 and @xmath49 ) while some are not ( @xmath165 and @xmath166 ) in the case where @xmath167 and @xmath168 .
situations such as @xmath175 and @xmath176 or @xmath177 and @xmath178 were studied too and yielded very similar results i.e. lesser improvements on the estimations of some parameters only .
note that when some coordinates of @xmath86 are valued to @xmath179 while some are valued to @xmath21 , the `` similarity '' between @xmath1 and @xmath0 is very weak .
the strength or weakness of the similarity between @xmath1 and @xmath0 can not be diagnosed directly from the posterior mean of @xmath86 itself but we will see that the estimations of the hyperparameters @xmath20 and @xmath90 may provide a partial answer to this question .
we also estimated the hyperparameters ( see section [ subsec : informativeprior ] for the specifications of @xmath180 ) when the informative prior was used .
let us first study the hyperparameter @xmath86 .
its coordinates seem correctly estimated for the ideal situation where @xmath163 as illustrated in the top row of figure [ fig : mean_k10et05 ] which shows the posterior error of @xmath86 . when @xmath167 and @xmath168 , the estimations obtained are of lesser quality as demonstrated in the bottom row of figure [ fig : mean_k10et05 ] : most of the seasonal similarity coefficients appear to be biased ( while the posterior standard deviation on each coordinate , not shown here , are greater than in the ideal situation ) .
these estimations may thus be used to quantify the closeness of the two datasets .
the estimation of the hyperparameter @xmath89 itself does not seem to provide a lot of information about the data : during our simulations , its mean value exhibited a lot of variability around the same value over the 300 replications for each of the five simulated scenarios and no reasonable conclusion could be drawn from it .
on the other hand , the estimation of the hyperparameter @xmath20 does reveal a bit of information about the two datasets @xmath1 and @xmath0 .
it is the mean of the coordinates of @xmath86 on the real axis , as can be seen in the definition of the informative prior in on page .
however its use remains somewhat limited in the sense that the parameters @xmath165 of the two datasets are most often very close ( meaning the coordinates of @xmath86 that correspond to them is likely close to 1 ) while other parameters may vary greatly .
hence even though @xmath20 provides information about the similarity between @xmath1 and @xmath0 , it can not be interpreted alone and has to be considered jointly with @xmath90 .
the left part of figure [ fig : seasonheat_qr ] shows the evolution of the posterior mean of @xmath20 as @xmath181 ranges over @xmath182}$ ] .
the estimation of the hyperparameter @xmath90 ( inverse - variance of the prior distribution on @xmath86 , see again ) does in fact reveal some information about the two datasets too .
it is a measure of dispersion of @xmath86 around @xmath20 , in the sense that the ( higher it is , the closer to @xmath20 the coordinates of @xmath86 should be . just like
@xmath20 is the mean of the coordinates of @xmath86 , @xmath90 is in fact their inverse - variance .
the right part of figure [ fig : seasonheat_qr ] shows a clear decline of @xmath90 when @xmath183 moves away from the ideal value 1 i.e. when the similarity between the datasets @xmath1 and @xmath0 decrease from strong to weak .
as we previously stated , the similarity between the two datasets has to be assessed simultaneously with @xmath20 and @xmath90 and not @xmath20 only : the mean @xmath20 could be close to 1 , possibly hinting at a perfect similarity between the two datasets , while the variance @xmath184 could be great which would then indicate huge differences between the two estimated sets of parameters for the two datasets .
( left ) and @xmath90 ( right , on a log scale ) for the informative prior ( abscissas have been jittered a bit to prevent overlapping , and different shades of grey are used to indicate the level of the estimated density ) .
300 replications for each value of @xmath185 tested . in black : the circles correspond to the averages , while the squares correspond to the 5% and 95% empirical quantiles . here @xmath167.,title="fig : " ]   ( left ) and @xmath90 ( right , on a log scale ) for the informative prior ( abscissas have been jittered a bit to prevent overlapping , and different shades of grey are used to indicate the level of the estimated density ) .
300 replications for each value of @xmath185 tested . in black : the circles correspond to the averages , while the squares correspond to the 5% and 95% empirical quantiles . here @xmath167.,title="fig : " ]     ( left , where @xmath167 ) and @xmath171 ( right , where @xmath186 ) tested . in black
: circles correspond to the averages , while squares and diamonds correspond to the 80% and 90% empirical quantiles of these ratios.,title="fig : " ]   ( left , where @xmath167 ) and @xmath171 ( right , where @xmath186 ) tested . in black : circles correspond to the averages , while squares and diamonds correspond to the 80% and 90% empirical quantiles of these ratios.,title="fig : " ]    _ prediction .
_ we compared the informative and the non - informative models using our comparison criterion defined in and computing the ratio between the two models for different values of @xmath187 and @xmath170 , @xmath169 and @xmath171 being both set to 1 .
the left part of figure [ fig : gain ] shows the results we obtained for @xmath173 and @xmath170 simultaneously set to the values @xmath188 .
note that since the results appeared to be approximately symmetric with regard to @xmath21 ( i.e. for values @xmath189 and @xmath190 ) , we only included one side of the graph in the present article .
on average , the bayesian informative model is a clear improvement over the bayesian non - informative one , its performances being maximised when the parameters @xmath191 and @xmath192 are identical ( which is the ideal situation ) .
the performances in prediction are obviously somewhat weakened when the difference between the parameters @xmath191 and @xmath192 grows greater , but the use of the informative hierarchical prior still leads to an average improvement of 15% over the non - informative model , as can be seen on figure [ fig : gain ] .
the results obtained when @xmath169 or @xmath171 are varying while the other coordinates of @xmath86 are fixed to 1 were very similar ( see for example the right part of figure [ fig : gain ] ) .
the long dataset @xmath1 needed for the construction of the informative prior corresponds to a specific population in france frequently referred to as `` non - metered '' because their electricity consumption is not directly observed by edf but instead derived as the difference between the overall electricity consumption and the consumption of the `` metered '' population . for this population
the data ranged from 07/01/2004 to 07/31/2010 .
we illustrate the benefit of choosing our informative prior to predict electricity load on short datasets .
we consider two short datasets : the first @xmath0 corresponds to the `` non - metered '' population for erdf , a wholly owned subsidiary of edf that manages the public electricity network for 95% of continental france .
this population roughly covers the same people that @xmath1 does , but not exactly .
the second dataset @xmath193 corresponds to a subpopulation of @xmath1 and represents around 50% of the total load of @xmath1 .      for this application ,
only the days for which no special tariffs are enforced were considered : the so - called ejp (  effacement jour de pointe
= peak tariff days ) were removed from the dataset beforehand to ensure the signal studied was consistent throughout time .
bank holidays ( including the day before and the day after to avoid any neighbourhood contamination effects ) , the summer holiday break ( august ) and the winter holiday break ( late december ) were also removed from the dataset for this first application , as we wanted to benchmark our method against others on a smooth and rather easy - going signal , so as not to put any one method at a disadvantage due to the signal s specificity .
the temperature considered in the model is the average temperature over france for the period of study , and the cooling threshold was chosen to be 16c throughout the 48 instants of the day .
we benchmarked our bayesian method with informative prior against four alternative methods , comparing their predictions on dataset @xmath0 in two configurations . roughly speaking , for our first experiment we estimated the model for @xmath0 over the period ranging from 12/01/2009 to 06/30/2010 and predicted the next 30 days ( same as the application presented section [ subsec : applihyper ] ) , while for our second experiment , we estimated the model for @xmath0 over the period ranging from 01/01/10 to 07/31/10 and predicted the previous 30 days .
we expect the first configuration to be the easy case and the second configuration to be the tough case , the signal being very smooth during summer and not so much during winter .
the figures shown in table [ tab.datasize ] summarise the exact lengths of the various datasets for both experiments .
the four alternative methods we benchmarked against our own bayesian informative method , relied on four different techniques : the first was the bayesian non - informative method that we exposed earlier in section [ subsec : noninformativeprior ] ( recall that it was meant to be an equivalent to the maximum likelihood approach ) , the second involved non - parametric estimation with kernels ( see * ? ? ? * ) , the third was a double exponential smoothing ( see * ? ? ? * ) and the fourth and last was an arima model .
note that for the second experiment , the data available obviously had to be time - reversed in order to apply some the last three alternatives methods since time - dependence plays an important role for them .
the arima model was automatically selected ( see * ? ? ?
* ) and was the best model with regard to the aic criterion .
.sample size ( in days ) of the datasets for both experiments .
[ cols="^,^,^,^",options="header " , ]     figure [ fig : mape_adelaide_mois ] shows the average error in prediction for each month while figures [ fig : mape_adelaide_instant_noninfo ] and [ fig : mape_adelaide_instant_info ] display the average error in prediction for each half hour ( from 00:00 to 23:30 ) .
it is important to note that the use of the non - informative prior leads to overfitting the model : the results presented in table [ tab : qualite_global ] show that as the estimation window goes smaller , the estimation error decreases while the prediction error grows very quickly . a close inspection of the posterior densities of the different parameters of the model revealed that the bias induced by the increasing lack of data is mainly seasonal : this is due to the seasonality coefficients of the model being overfit . choosing the informative prior over the non - informative
prior makes the estimation and prediction of the model more robust with regards to the lack of data .
the informative prior especially improves the quality of the predictions when the lack of data is severe : it provides reasonable forecasts even in the worst scenario considered here , where only 6 months of estimation were used for 6 months of prediction .
in this situation , estimation ( from 07/01/2010 to 12/31/2010 ) and prediction ( from 01/01/2011 to 06/30/2011 ) are performed on non - overlapping areas of the calendar : the informative prior makes up for the unavailable data and prevents the model from overfitting on the second half of the calendar , while the non - informative prior does not and consequently leads to heavily biased predictions over the first half of the calendar .
the two mcmc algorithms presented below were developed because direct simulations from the posterior distribution were not possible .
the justifications are given after the algorithms themselves .
notice that the full conditional distributions of all the parameters but the threshold @xmath49 appear to be common distributions in both cases , due to the presence of multiple semi - conjugacy situations .
we used a metropolis - within - gibbs algorithm ( see * ? ? ?
* , for a quick description ) based on gibbs sampling steps for every parameter but @xmath49 for which we used a metropolis - hasting step based on a gaussian random walk proposal . the algorithm corresponding to the non - informative prior is detailed first since it is the simplest of the two .
we define the ( commutative and associative ) operator @xmath194 as @xmath195^{-1}(\sigma_1^{-1 } \mu_1 + \sigma_2^{-1 } \mu_2 ) \\ ~[\sigma_1^{-1 } + \sigma_2^{-1}]^{-1 } \end{array}\right)\end{aligned}\ ] ] for any vectors @xmath196 and @xmath197 in @xmath198 , for any symmetric positive definite matrices @xmath199 and @xmath200 of size @xmath201 .
[ lemmaconj ] let @xmath202 and @xmath203 be two random truncated gaussian vectors in @xmath198 @xmath204 and denote @xmath205 and @xmath206 their respective densities , then @xmath207 is integrable .
let furthermore @xmath208 be a random variable with density @xmath209 , then @xmath208 has truncated gaussian distribution @xmath210 where @xmath211and this result easily extends to any finite number of random truncated ( or not ) gaussian vectors .
[ lemmacond ] let @xmath212 be a random gaussian vector in @xmath198 @xmath213 } \sim { \mathcal{n}}\left({\left[\begin{array}{c}\mu_1 \\ \mu_2\end{array}\right ] } , { \left[\begin{array}{cc}r & s \\ s^\prime & t\end{array}\right]}^{-1}\right)\ ] ] and @xmath202 and @xmath203 the projections of @xmath212 over its @xmath214 first and @xmath215 last coordinates ( @xmath216 ) .
the conditional distribution of @xmath202 with regard to @xmath203 is then gaussian @xmath217    [ lemmalinearreg ] let @xmath212 and @xmath208 be two random vectors respectively in @xmath198 and @xmath218 such as the conditional distribution of @xmath208 with regard to @xmath212 is gaussian @xmath219 with @xmath55 matrix of size @xmath220 that has full rank @xmath221 , and let @xmath222 be a fixed vector in @xmath218 .
the conditional distribution of @xmath212 with regard to @xmath208 is then gaussian too @xmath223^{-1 } m^\prime ( y - z ) , \sigma^2 m^\prime m\right).\ ] ]    denoting @xmath224 , straightforward algebra leads immediately to @xmath225^\prime \sigma^2 m^\prime m \big[(m^\prime m)^{-1}m^\prime w - x\big ] \\ & \quad\quad - \big[(m^\prime m)^{-1}m^\prime w \big]^\prime \sigma^2 m^\prime m \big[(m^\prime m)^{-1}m^\prime w\big ] \\ & \quad\quad + w^\prime\sigma^2 i_n w\end{aligned}\ ] ] where the two last terms on the right hand side of the equation do not depend on @xmath212 .      in the lines below
, we give the different steps of the mcmc algorithm we used to ( approximately ) simulate @xmath226 according to the posterior distribution @xmath227 corresponding to the non - informative prior we presented earlier .
the algorithm goes as follows :    1 .
initialise @xmath228 such that @xmath229 2 .   for @xmath230 ,
repeat 3 .
simulate @xmath231 i.e. @xmath232 4 .
simulate @xmath233 i.e. @xmath234 5 .
simulate @xmath235 i.e. @xmath236 6 .
simulate @xmath237 i.e. @xmath238 7 .
simulate @xmath239 , simulate @xmath240}$ ] and define @xmath241 * define @xmath242 if @xmath243 * or @xmath244 otherwise + where the covariance matrix @xmath245 used in this last metropolis - hastings step is first estimated over a burn - in phase ( the iterations coming from this phase are discarded ) , and then fixed to its estimated value `` asymptotically optimally rescaled '' for the final run by a factor @xmath246 ( as recommended for gaussian proposals in section 2 of * ? ? ?
* ) .    the justifications for each full conditional distribution used in the gibbs sampling steps , including the explicit expressions of @xmath247 , are now given .
lemma [ lemmalinearreg ] is a key element to these justifications .
_ full conditional distribution of @xmath174 .
_ denote @xmath248 the size of the vector @xmath174 , and @xmath249 the vector @xmath2 from which the coordinates corresponding to @xmath174 have been removed .
the full conditional distribution of @xmath174 can directly be deduced from both the prior and the likelihood contributions to it .
let us first observe that , since the prior distribution we are using on @xmath174 is flat , the full conditional distribution of @xmath174 is in fact proportional to the likelihood function ( seen as a function of @xmath174 ) . now considering the likelihood contribution , we write @xmath250    let now @xmath251 be the diagonal matrix whose diagonal coefficients are given by @xmath252 let @xmath253 be the vector whose coordinates are given by @xmath254 and denote @xmath255 the matrix @xmath256 .
we can now rewrite @xmath257 and get @xmath258    using lemma [ lemmalinearreg ] , it is then straightforward to see that the full conditional distribution of @xmath174 is gaussian @xmath259 where @xmath260^{-1 } m_\alpha^\prime ( y - z_\alpha)\\ \sigma^2 m_\alpha^\prime m_\alpha \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath165 . _ using similar arguments , we obtain the full conditional distribution of @xmath165 .
namely , denoting @xmath261 the vector whose coordinates are given by @xmath262 and calling @xmath263 where @xmath264 is the diagonal matrix whose diagonal is @xmath265 , we obtain the truncated gaussian distribution @xmath266 where @xmath267^{-1 } m_\beta^\prime ( y - z_\beta)\\ \sigma^2 m_\beta^\prime m_\beta \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath166 . _
using again similar arguments , we obtain the full conditional distribution of @xmath166 .
namely , denoting @xmath268 the vector whose coordinates are given by @xmath269 and calling @xmath270 the vector whose coordinates are @xmath142 we obtain the gaussian distribution @xmath271 where @xmath272^{-1 } m_\gamma^\prime ( y - z_\gamma)\\ \sigma^2 m_\gamma^\prime m_\gamma \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath91 .
_ no calculations are required , as we immediately identify an inverse - gamma distribution from .      in the lines below we give the different steps of the mcmc algorithm we used to ( approximately )
simulate @xmath226 according to the posterior distribution @xmath227 corresponding to the informative prior we presented earlier .
the algorithm goes as follow :    1 .
initialise @xmath228 such that @xmath229 2 .   for @xmath230 ,
repeat 3 .
simulate @xmath273 @xmath232 4 .
simulate @xmath274 @xmath275 5 .
simulate @xmath276 @xmath277^{-1}(\sigma_q^{-2 } + r \sum_{i=1}^d k_i ) , [ \sigma_q^{-2}+rd]^{-1}\right)\ ] ] 6 .
simulate @xmath278 @xmath279 7 .
simulate @xmath280 @xmath281 8 .
simulate @xmath282 @xmath283 9 .
simulate @xmath284 @xmath285 10 .
simulate @xmath286 @xmath287 11 .
simulate @xmath288}$ ] and define @xmath241 * define @xmath242 if @xmath289 * or @xmath244 otherwise    where the covariance matrix @xmath245 used in the metropolis - hastings step is first estimated over a burn - in phase , and then fixed to its rescaled estimated value for the real run as in the non - informative approach .    the justifications for each full conditional distribution used in the gibbs sampling steps , including the explicit expressions of @xmath290 and @xmath291 , are now given . to derive these full conditional distributions
, we will make use of the technical lemmas [ lemmaconj ] , [ lemmacond ] and [ lemmalinearreg ] presented earlier .
_ full conditional distribution of @xmath174 .
_ denote @xmath248 the size of the vector @xmath174 , and @xmath249 the vector @xmath2 from which the coordinates corresponding to @xmath174 have been removed .
the full conditional distribution of @xmath174 can directly be deduced from both the prior and the likelihood contributions to it .
denote @xmath292 , and write the full conditional distribution of @xmath174 as @xmath293 where @xmath294 is the contribution of the likelihood ( seen as a function of @xmath174 to the full conditional distribution ) and @xmath295 is the contribution of the prior ( seen as a function of @xmath174 ) .
we prove that @xmath296 and @xmath297 both correspond to gaussian distributions before using lemma [ lemmaconj ] to combine them into yet another gaussian distribution .    1 .
let us first consider the prior contribution @xmath297 .
recall first that @xmath174 only appears in the following component of the prior @xmath298 which directly implies that @xmath299 denote @xmath300 , @xmath301 and denote @xmath302 and @xmath303 the vectors resulting from the extractions of the coordinates corresponding to @xmath174 and @xmath304 from @xmath257 . finally denote @xmath305 the matrix resulting from the extraction of the rows and columns both corresponding to @xmath174 of @xmath306 and denote @xmath307 the one resulting from the extraction of the rows corresponding to @xmath174 and columns corresponding to @xmath304 of @xmath306 . using lemma [ lemmaconj ] ( and reordering indexes if necessary ) it is straightforward that @xmath295 is proportional to the density of a gaussian distribution @xmath308 2 .
let us now consider the likelihood contribution .
using exactly the same notations that we used for the full conditional distribution of @xmath174 for the algorithm associated with the non - informative approach we immediately find that @xmath294 is proportional to the density of a gaussian distribution @xmath309^{-1 } m_\alpha^\prime ( y - z_\alpha ) , \sigma^2 m_\alpha^\prime m_\alpha)\end{aligned}\ ] ] just as in .
3 .   with the help of lemma [ lemmaconj ] and using the two results above
, we can now deduce the posterior conditional distribution of @xmath174 and obtain the gaussian distribution @xmath310 where @xmath311^{-1 } m_\alpha^\prime ( y - z_\alpha)\\ \sigma^2 m_\alpha^\prime m_\alpha \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath165 .
_ using similar arguments , we obtain the full conditional distribution of @xmath165 .
namely , keeping the notation introduced to derive , and combining the prior and the likelihood contributions together with lemma [ lemmaconj ] we obtain the truncated gaussian distribution @xmath312 where @xmath313^{-1 } m_\beta^\prime ( y - z_\beta)\\ \sigma^2 m_\beta^\prime m_\beta \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath166 . _ using again similar arguments , we obtain the full conditional distribution of @xmath166 .
namely , keeping the notation introduced to derive , and combining the prior and the likelihood contributions together with lemma [ lemmaconj ] we obtain the gaussian distribution @xmath314 where @xmath315^{-1 } m_\gamma^\prime ( y - z_\gamma)\\ \sigma^2 m_\gamma^\prime m_\gamma \end{array}\right).\end{aligned}\ ] ]    _ full conditional distribution of @xmath86 .
_ we denote @xmath316 and first notice that @xmath317 .
using the definition of the informative prior and lemma [ lemmaconj ] , we then immediately derive @xmath318 where @xmath319    _ full conditional distribution of @xmath320 and @xmath91 . _
no calculations are required , as we respectively identify a gamma distribution , a gaussian distribution , a gamma distribution , and an inverse - gamma distribution from .
the authors would like to thank adlade priou for collecting a part of the data as well as the corresponding results , and virginie dordonnat for the insightful discussions .