the recent discovery  @xcite of a new particle with properties consistent with a standard model ( sm ) higgs boson is clearly the most significant news from the large hadron collider ( lhc ) .
this discovery was supported by evidence for a higgs boson found by the cdf and d0 collaborations at the tevatron @xcite and completes our picture of the sm .
however , the sm leaves many fundamental questions open  perhaps the most pressing issue being that the sm does not explain the value of the electroweak scale ,   the higgs mass , itself . clearly , a prime goal after the discovery is to thoroughly test the sm nature of this higgs - like signal .    with the measurements in various channels , a comprehensive study of the properties of the higgs - like state becomes possible and has the potential for revealing whether or not the higgs sector is as simple as envisioned in the sm . in a recent study  @xcite , we analyzed the extent to which the results from the lhc and tevatron , as published by the end of 2012 , constrain the couplings of the higgs boson - like state at @xmath0 . to this end
we assumed that only sm particles appear in the higgs decays , but tree - level higgs couplings to the up - quarks , down - quarks and vector bosons , relative to the sm are free parameters .
moreover , we considered the case that new particles appearing in loops modify the effective higgs couplings to gluons and/or photons .
we found that the sm expectation is more than @xmath7 away from fits in which : a ) there is some non - sm contribution to the @xmath8 coupling of the higgs ; or b ) the sign of the top quark coupling to the higgs is opposite that of the sm . in both these cases good
fits with @xmath9-values @xmath10 can be achieved .
since option b ) is difficult to realize in realistic models , it would seem that new physics contributions to the effective couplings of the higgs are preferred .    in this paper
we carry our work a step further and investigate the extent to which current data constrain invisible decays (  @xmath11 , where @xmath12 is the lightest susy particle ) or undetected decays ( such as @xmath13 , where @xmath14 is a light cp - odd , perhaps singlet scalar ) of the higgs - like particle . with a small @xmath15 of about @xmath16 . for the global fits , there is no difference between invisible and yet undetected decays , and we will use the term  invisible " for both , genuine invisible and merely undetected . when talking about specific signatures , like monojets + missing energy , it is however important to make the distinction  in this case we will use `` invisible '' in the strict sense . ]
our parametrization and fitting procedure is the same as in @xcite , where we introduced scaling factors @xmath17 relative to sm couplings .
we treat the couplings to up - type and down - type fermions , @xmath18 and @xmath19 , as independent parameters ( but we only consider the case @xmath20 , and we assume that the @xmath21 are family universal ) .
moreover , we assume a custodial symmetry in employing a single @xmath22 .
in addition to the tree - level couplings given above , the @xmath23 has couplings to @xmath24 and @xmath8 that are first induced at one loop and are completely computable in terms of @xmath18 , @xmath19 and @xmath1 if only loops containing sm particles are present .
we define @xmath25 and @xmath26 to be the ratio of these couplings so computed to the sm (  @xmath27 ) values .
however , in some of our fits we will also allow for additional loop contributions @xmath28 and @xmath29 from new particles ; in this case @xmath30 and @xmath31 .
limits on `` invisible '' higgs decays from global fits to lhc data were obtained previously in  @xcite and within the mssm in  @xcite . since then , much more data has become available .
we therefore find it worthwhile to re - investigate the status of invisible decays .
moreover , we consider more general deviations from sm couplings than the previous works .    the possibility of probing directly the ( genuine ) invisible branching fraction of the higgs in various channels at the lhc has been investigated already some time ago  @xcite .
it was revisited more recently in light of the 125  gev higgs signal at the lhc in  @xcite .
the channels considered were monojets or 2 jets plus missing energy .
associated production , leading to high-@xmath32 @xmath33 s plus missing energy .
however , stronger limits were obtained from monojet searches . ]
these make it possible to constrain the product of the higgs production cross section in the gluon
gluon fusion ( ggf ) , or vector boson fusion ( vbf ) and vh associated production modes , times the invisible branching fraction . specifically , they put constraints on @xmath34 and @xmath35
. we will not include these direct limits in our fits but will comment on the impact of these probes on the valid parameter space . probing yet undetected
@xmath13 decays ( with the pseudoscalars further decaying in various channels ) was discussed in  @xcite  results depend strongly on the manner in which the @xmath14 decays .
the results of our global fits are presented in section  2 . in section  3 , we discuss further probes of invisible or undetected decays . in the case of higgs
decays into dm particles , a strong interplay with direct dm searches arises .
this is discussed in section  4 .
a summary and conclusions are given in section  5 .
distributions for the branching ratio of invisible higgs decays . the full , dashed , and dotted lines correspond , respectively , to the cases of 1 ) sm couplings , 2 ) arbitrary @xmath28 and @xmath29 , and 3 )  deviations of @xmath36 from unity . in addition
, we show as dash - dotted line the variant of case 3 ) with @xmath37 and @xmath38 .
[ fig : onedim ] ]    as mentioned , we consider several cases in our analysis .
we start the discussion with the simplest case , case 1 ) , of an sm higgs boson augmented by invisible / undetected decays ,  we require that all couplings be equal to the sm values ( @xmath39 ) but allow for ( yet ) invisible decays of the higgs state in addition to the normal decays to sm particles .
we first recall that , for the most part , the observed rates in many production - decay channels are not far from the sm - predictions and , in particular , are not suppressed .
in addition , there are a few final states that have a somewhat enhanced rate relative to the sm , most notably the @xmath8 final state . since invisible decays reduce the branching ratio to the ( visible ) sm final states , it is to be expected that @xmath40 is strongly limited .
this is confirmed in our fit .
we find that @xmath41 ( @xmath42 ) at @xmath7 ( @xmath43 ) .
the @xmath44 distribution as function of the invisible rate for case 1 ) is shown as the solid line in fig .
[ fig : onedim ] .
the @xmath44 at the minimum is @xmath45 ,  the same as for the sm  @xcite . with 20 degrees of freedom ( d.o.f . )
this corresponds to a @xmath9-value of @xmath46 , as compared to a @xmath9-value of @xmath47 for the pure sm ( 21  d.o.f . ) .
let us now turn to case 2 ) , where we allow for extra loop contributions to the @xmath24 and @xmath8 couplings of the higgs boson , denoted @xmath28 and @xmath29 , respectively .
such loops would involve particles beyond those present in the sm .
if these bsm particles receive mass from the higgs mechanism , then they will give contributions to @xmath28 and @xmath29 that approach a constant value as their mass increases in size  they do not decouple .
allowing for arbitrary @xmath28 and @xmath29 the minimum @xmath44 decreases from the sm fit value of @xmath48 to @xmath49 with the best values of @xmath50 and @xmath51 for a non - zero value of the invisible rate , @xmath52 .
the one - dimensional ( 1d ) distribution of @xmath53 as a function of @xmath40 , after profiling over @xmath28 and @xmath29 , is plotted as the dashed line in fig .
[ fig : onedim ] .
the 95% cl upper limit on the invisible rate increases dramatically to @xmath54 .
larger @xmath40 can be accommodated by increasingly large values of @xmath28 so that the overall production rates in sm final states from @xmath24 fusion processes remain the same .
rates in sm final states for vector boson fusion ( vbf ) induced processes will decline somewhat and this is the primary reason for the increase in @xmath44 with increasing @xmath40 .
the @xmath55 , @xmath7 and @xmath43 ranges in the @xmath28 versus @xmath29 plane are shown in the top plot in fig .
[ fig : delta - cg - cp ] .
it is interesting to compare to the case of arbitrary @xmath28 and @xmath29 without invisible decays , for which the @xmath55 and @xmath7 ranges are shown as black and gray contours in fig .
[ fig : delta - cg - cp ] . allowing for invisible decays , we find @xmath56 , @xmath57 ( @xmath58 ) as compared to @xmath59 , @xmath60 ( @xmath61 ) when invisible decays are absent .
contours of @xmath40 versus @xmath28 and versus @xmath29 are shown in the bottom row of fig .
[ fig : delta - cg - cp ] .
we see that , as mentioned , larger @xmath40 can be accommodated by increasing @xmath28 . on the other hand , larger negative @xmath28 , as for example induced by light stops with large mixing , disfavors invisible decays . finally , @xmath40 is rather insensitive to @xmath29 .
contours obtained by allowing for additional loop contributions @xmath62 and @xmath63 to the effective couplings of the higgs to gluons and photons .
the red , orange and yellow areas ( black , grey and light grey areas in greyscale print ) are the 68% , 95% and 99.7%  cl regions , respectively .
the black and grey ellipses in the top plot show the 68% and 95% cl contours when invisible decays are absent .
we find @xmath64 , @xmath65 and @xmath66 ( where the errors correspond to 1d profiling ) . at the best fit point , marked as a white star , we find @xmath67 ( for 18  d.o.f . ) .
[ fig : delta - cg - cp ] , title="fig : " ] +    contours obtained by allowing for additional loop contributions @xmath62 and @xmath63 to the effective couplings of the higgs to gluons and photons .
the red , orange and yellow areas ( black , grey and light grey areas in greyscale print ) are the 68% , 95% and 99.7%  cl regions , respectively .
the black and grey ellipses in the top plot show the 68% and 95% cl contours when invisible decays are absent .
we find @xmath64 , @xmath65 and @xmath66 ( where the errors correspond to 1d profiling ) . at the best fit point , marked as a white star , we find @xmath67 ( for 18  d.o.f . ) .
[ fig : delta - cg - cp ] , title="fig : " ] contours obtained by allowing for additional loop contributions @xmath62 and @xmath63 to the effective couplings of the higgs to gluons and photons .
the red , orange and yellow areas ( black , grey and light grey areas in greyscale print ) are the 68% , 95% and 99.7%  cl regions , respectively .
the black and grey ellipses in the top plot show the 68% and 95% cl contours when invisible decays are absent .
we find @xmath64 , @xmath65 and @xmath66 ( where the errors correspond to 1d profiling ) . at the best fit point , marked as a white star , we find @xmath67 ( for 18  d.o.f . ) .
[ fig : delta - cg - cp ] , title="fig : " ]      in case 3 ) , we allow for arbitrary @xmath36 while varying @xmath40 . we do not allow for extra contributions from bsm particles to the @xmath24 and @xmath8 higgs couplings ,  we assume @xmath68 and @xmath69 .
it is interesting to consider several sub - cases .    for general @xmath36 ( limited however to the range @xmath70 ) , we find @xmath71 , @xmath72 and @xmath73 .
the best fit point has @xmath74 for 17 d.o.f .  and is basically the same as without invisible decays .
a large @xmath40 can yield a small @xmath75 when simultaneously increasing both @xmath76 ( so as to increase the @xmath24 production rates ) and @xmath1 ( so as to increase the vbf production rates ) in order to compensate for the decreasing branching ratios to sm particle final states .
( we also need to increase @xmath77 in order to have enough decays into @xmath78 and @xmath79 . ) as in the fits of  @xcite , the minimum @xmath44 is achieved for negative @xmath18 , something that is very difficult in the context of most theoretical models , see the comment at the end of this section .
in addition , the large values of @xmath80 required for a good fit with large @xmath40 imply an enhancement of the isospin @xmath81 cross section , which may be achieved if the higgs state at @xmath82 125 gev mixes with higher su(2 ) representations  @xcite .
( within extensions of the higgs sector by an arbitrary number of su(2 ) doublets and/or singlets only , one generally obtains @xmath83 . )
allowing for large negative @xmath18 and large positive @xmath1 ( up to @xmath84 , @xmath85 ) , @xmath40 can only be constrained to @xmath86 at 95% cl , see fig .
[ fig : cu - cd - cv ] .     allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 , but without extra loop contributions ,  @xmath87 .
same color code as in fig .
[ fig : delta - cg - cp ] . allowing up to 100% deviations in @xmath18 and @xmath1 ,
the @xmath7 ( @xmath43 ) limit is @xmath88 ( 0.92 ) .
[ fig : cu - cd - cv ] , title="fig : " ] allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 , but without extra loop contributions ,  @xmath87 .
same color code as in fig .
[ fig : delta - cg - cp ] . allowing up to 100% deviations in @xmath18 and @xmath1 ,
the @xmath7 ( @xmath43 ) limit is @xmath88 ( 0.92 ) .
[ fig : cu - cd - cv ] , title="fig : " ]     but restricting either @xmath89 ( left ) or @xmath90 ( right ) .
[ fig : cu - cd - cv-2 ] , title="fig : " ] but restricting either @xmath89 ( left ) or @xmath90 ( right ) .
[ fig : cu - cd - cv-2 ] , title="fig : " ]    restricting the fit to @xmath89 , as relevant for a higgs sector consisting of doublets+singlets , the minimum @xmath44 remains at @xmath91 but the @xmath53 increases rapidly with the invisible rate .
we find @xmath92 in this case , and @xmath93 at 95%  cl .
obviously , imposing @xmath94 not only greatly restricts the production cross sections that can be achieved in vector boson fusion but also restricts the two - photon partial width of the higgs and therefore decreases the value of @xmath40 that can allow reasonable consistency with the experimental observations .
@xmath40 versus @xmath18 for @xmath89 is shown in the left plot in fig .
[ fig : cu - cd - cv-2 ] .
another interesting case is to require @xmath90 while allowing @xmath80 . in this case , the minimum @xmath44 point is characterized by @xmath95 , @xmath96 , @xmath97 , and the fit worsens to @xmath98 .
further , @xmath40 can only be constrained to @xmath99 at 95%  cl . for illustration , see the right - hand - side plot in fig .
[ fig : cu - cd - cv-2 ] .
perhaps most interesting from the theoretical point of view is the case in which @xmath37 and @xmath89 is required . with these constraints , @xmath100
(  not far from the sm fit value of @xmath48 ) and the 95% cl limit on @xmath40 is @xmath101 . the @xmath53 distribution for this case is shown as dash - dotted line in fig .
[ fig : onedim ] . for @xmath40 versus @xmath18 and @xmath1 , see fig .
[ fig : brinv - cu - cd - pos - cvle1 ] .
contours obtained by requiring @xmath89 and @xmath37 .
same color code as in previous figures .
[ fig : brinv - cu - cd - pos - cvle1 ] , title="fig : " ]    contours obtained by requiring @xmath89 and @xmath37 .
same color code as in previous figures .
[ fig : brinv - cu - cd - pos - cvle1 ] , title="fig : " ]     allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 . in the top row of plots ,
@xmath38 but @xmath102 may have either sign . in the bottom row ,
@xmath89 and @xmath37 .
color code _ etc .
_  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ]   allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 . in the top row of plots ,
@xmath38 but @xmath102 may have either sign . in the bottom row ,
@xmath89 and @xmath37 .
color code _ etc .
_  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ]   allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 . in the top row of plots , @xmath38 but @xmath102 may have either sign .
in the bottom row , @xmath89 and @xmath37 .
color code _ etc .
_  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ] +   allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 .
in the top row of plots , @xmath38 but @xmath102 may have either sign . in the bottom row ,
@xmath89 and @xmath37 .
color code _ etc . _  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ]   allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 . in the top row of plots ,
@xmath38 but @xmath102 may have either sign . in the bottom row ,
@xmath89 and @xmath37 .
color code _ etc .
_  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ]   allowing for deviations of @xmath18 , @xmath19 , @xmath1 from 1 . in the top row of plots ,
@xmath38 but @xmath102 may have either sign . in the bottom row ,
@xmath89 and @xmath37 .
color code _ etc .
_  as in previous figures .
[ fig : cu - cd - cv - par ] , title="fig : " ]    the relevant 2d correlations between parameters , illustrating the discussion above , are shown in fig .  [
fig : cu - cd - cv - par ] for : the fit requiring @xmath38 but allowing arbitrary signs for @xmath102 ( top row ) ; and the fit requiring both @xmath89 and @xmath37 ( bottom row ) . in order to see the impact of invisible decays on the coupling fits ,
we have superimposed the @xmath55 and @xmath7 regions from @xcite obtained for @xmath103 .
let us end this section with a comment on @xmath104 . a negative sign of @xmath105while maintaining a positive sign of @xmath106is actually not easy to achieve .
if the top quark and higgs bosons are considered as fundamental fields , it would require that the top quark mass is induced dominantly by the vev of at least one additional higgs boson which is not the higgs boson considered here , and leads typically to various consistency problems as discussed , , in @xcite .
truly invisible higgs decays can be probed at the lhc in monojet searches in either the ggf mode where a gluon is radiated from the initial state , or in vbf when one of the jets is missed .
invisible decays can also be probed in @xmath107 associated production with @xmath108 . in  @xcite , sensitivity to the monojet searches is phrased in terms of limits on @xmath109 a 95% cl upper limit of @xmath110 was obtained using the cms monojet analysis at @xmath111  tev and @xmath112  fb@xmath113  @xcite .
the relative contributions of the gluon and vector boson fusion production mechanisms were assumed to be the same as in the sm after the analysis cuts . of course ,
this need not apply if @xmath36 are allowed to vary .
( assuming only one production channel , the 95% cl upper limits are @xmath114 and @xmath115 . ) the projected limit for @xmath116  tev and @xmath117  fb@xmath113 is @xmath118 .     versus @xmath119 , on the left for the fit of @xmath28 and @xmath29 , on the right for the fit of @xmath36 with @xmath38 .
the red , orange and yellow regions are 68% , 95% and @xmath120% cl regions , respectively .
the dark ( light ) blue bands show the constraints from the monojet search , @xmath121 ( @xmath122 ) at 95% cl , with the bands obtained by varying @xmath29 ( left plot ) or @xmath1 ( right plot ) within their fitted @xmath7 ranges .
[ fig : monojet ] , title="fig : " ] versus @xmath119 , on the left for the fit of @xmath28 and @xmath29 , on the right for the fit of @xmath36 with @xmath38 .
the red , orange and yellow regions are 68% , 95% and @xmath120% cl regions , respectively .
the dark ( light ) blue bands show the constraints from the monojet search , @xmath121 ( @xmath122 ) at 95% cl , with the bands obtained by varying @xmath29 ( left plot ) or @xmath1 ( right plot ) within their fitted @xmath7 ranges . [
fig : monojet ] , title="fig : " ]    since the signals are also proportional to the possibly non - standard higgs production cross section @xmath123 in the ggf mode or @xmath124 in the vbf mode , @xmath34 and @xmath125 .
upper limits on @xmath126 will thus constrain @xmath40 as function of @xmath119 , as shown in fig .  [
fig : monojet ] for case 2 ) on the left and case 3 ) with @xmath89 on the right .
these plots should be compared to @xmath40 versus @xmath28 in fig .
[ fig : delta - cg - cp ] and @xmath40 versus @xmath18 in fig .
[ fig : cu - cd - cv-2 ] .
the dark ( light ) blue bands indicate @xmath127 ( @xmath122 ) , with the band obtained by varying @xmath29 or @xmath1 within @xmath7 . as can be seen , the monojet searches are already quite complementary in constraining invisible higgs decays when there is a large increase in the production cross section .
another analysis  @xcite considered searching for invisible higgs in the 2 jets and missing @xmath128 channel showing that a @xmath129 signal could be observed at 8  tev for @xmath130  fb@xmath113 for an sm production cross section provided @xmath131 , while the lhc at 14  tev could probe @xmath132 with @xmath133  fb@xmath113 . since the 2 jets @xmath134 channel is dominated by vbf production ( 86% after analysis cuts ) , this can be useful to constrain the cases with @xmath135 .
for example , @xmath136 @xmath137 could be probed at 8  tev for @xmath138 and @xmath139 @xmath140 , thus covering a large fraction of the currently allowed parameter space at large values of @xmath141 in fig .  3 and fig .  4 ( right panel ) .
let us finally comment on decays that may in principle be detectable .
the ability of the lhc to probe for higgs decays into light pseudoscalars , @xmath13 , depends on the decays of the @xmath14 s .
the most likely @xmath14 decays are @xmath142 , dominant for @xmath143 , and @xmath144 , dominant for @xmath145 .
a review , with detailed referencing , of the possibilities for the lhc in various production modes in the cases of these decays is given in @xcite . in two - higgs - doublet models @xmath146 ( where @xmath147 is a light quark ,  @xmath148 or @xmath149 ) can be significant if @xmath150 or @xmath151 .
lhc sensitivity in this case has been examined for @xmath107 production in  @xcite . in all the different @xmath14 decay scenarios
pretty much full lhc luminosity , @xmath152 at @xmath153 is required to place strong limits (  @xmath154 at 95% cl ) on @xmath13 decays .
another potentially interesting decay channel , that may have escaped observation , is @xmath155 , with a soft photon .
this may arise for instance in @xmath156 decays followed by @xmath157 , where @xmath158 denotes a gravitino @xcite or a goldstino @xcite .
assuming that the invisible particle which the higgs potentially decays into is the dark matter of the universe , the lhc bounds on @xmath40 can be turned into bounds on the dm scattering off nucleons , mediated by higgs exchange , _ cf .
_  @xcite .
these bounds are often much stronger than the current limits from xenon100 for @xmath159  gev ( _ i.e. _  @xmath160 ) .
both the invisible width of the higgs and the spin - independent cross - section for scattering on protons depend on the square of the higgs  dm  dm coupling @xmath161 .
if the dm is a majorana fermion , @xmath162 , the invisible width arising from @xmath163 decays is given by @xmath164 where @xmath165 and @xmath166 is defined by @xmath167 . in case of the dm being a real scalar , @xmath168 , we have @xmath169 and @xmath170    the spin - independent cross - section for scattering on a nucleon , considering only the higgs exchange diagram , can then be directly related to the invisible width of the higgs : @xmath171 ^ 2 \ ,      \label{eq : sigsi}\ ] ] with @xmath172 for a majorana fermion and @xmath173 for a real scalar ; @xmath174 is the reduced mass and @xmath175 are the quark ( gluon ) coefficients in the nucleon .
we take the values @xmath176 , @xmath177 , and @xmath178 from an average of recent lattice results  @xcite .
the gluon and heavy quark ( @xmath179 ) coefficients are related to those of light quarks , and @xmath180 at leading order . since the contribution of heavy quarks to the scattering amplitude originates from their contribution to the @xmath181 coupling
, we write the effect of @xmath28 , the last term in eq .
( [ eq : sigsi ] ) , in terms of an additional top quark contributing to the @xmath181 coupling ; numerically @xmath182 with only the sm top - quark contribution taken into account for computing @xmath183 .    for the numerical evaluation of @xmath184 , we use micromegas  @xcite in which the relation between the heavy quark coefficients and the light ones are modified by qcd corrections .
this amounts to taking @xmath185     as a function of the mass of the dm particle , for @xmath186 ( from bottom to top ) for the case of a majorana @xmath162 ( left panel ) or a real scalar @xmath168 ( right panel ) when @xmath187 and @xmath188 ,  an sm higgs plus invisible decays .
the red dashed curves show the xenon100 exclusion limit .
[ brinv - sigmasi ] , title="fig : " ] as a function of the mass of the dm particle , for @xmath186 ( from bottom to top ) for the case of a majorana @xmath162 ( left panel ) or a real scalar @xmath168 ( right panel ) when @xmath187 and @xmath188 ,  an sm higgs plus invisible decays .
the red dashed curves show the xenon100 exclusion limit .
[ brinv - sigmasi ] , title="fig : " ]    the results for @xmath184 versus the dm mass and for different @xmath40 are displayed in fig .
[ brinv - sigmasi ] for a majorana fermion ( left panel ) and a real scalar ( right panel ) assuming @xmath187 .
as can be seen , for a majorana fermion the current xenon100 limits  @xcite exclude , for example , @xmath189 when @xmath190 . for scalar dm , the cross sections are larger , and xenon100 excludes @xmath189 for @xmath191  gev .
these limits become much stronger when @xmath105 and/or @xmath192 are large provided they have the same sign .
further , these limits become stronger when we include a non - zero value of @xmath193 . for example ,
for @xmath194 we find that @xmath184 increases by a factor 1.8 as compared to the case @xmath195 for any given value of @xmath40 .
this increase is due in part to the new contribution in eq .
( [ eq : sigsi ] ) and in part because a larger coupling of the dm to the higgs is necessary to maintain a constant value of @xmath40 .
note that imposing universality of quark couplings to the higgs has an impact on our predictions for @xmath184 since all quark flavors contribute to this observable , whereas universality plays basically no role for higgs decays as only the third generation is important .    when @xmath196 and @xmath197 , corresponding to the best fit for case 3 ) , there is a destructive interference between the @xmath198-type and @xmath199-type quark contributions such that @xmath184 is much below the current limit .
note however that a negative sign of @xmath105 would require that @xmath106 is induced dominantly by the vev of a higgs boson which is not the higgs boson considered here ; if such a higgs boson also couples to dark matter it could then contribute significantly to the si cross section . without a complete model for the higgs sector
it is therefore difficult to make generic predictions in this case .
when the dm candidate is a dirac fermion and one assumes the same amount of matter and anti - matter in the early universe , the results for @xmath184 are simply a factor @xmath200 lower then those obtained in the majorana case .
however if this fermion also couples to the @xmath33 , this gives an additional positive contribution to @xmath184 , thus leading to stronger constraints from direct detection experiments .
similar arguments hold for the case of a complex scalar , as compared to a real scalar .
assuming that the 125  gev state observed at the lhc is , indeed , a higgs boson a very important question is whether or not it has decays to non - sm particles that escape undetected .
truly invisible decays include , for instance ,
@xmath201  lsp+lsp ( the lsp being the lightest supersymmetric particle in r - parity conserving supersymmetry and a dm candidate ) while undetected , but not intrinsically invisible , decays are typified by @xmath13 where @xmath14 is a light pseudoscalar of an extended higgs sector . in these and many other cases , `` invisible '' @xmath23 decays provide a portal to bsm physics that might prove hard to detect in any other way .    in this paper , we have assessed the extent to which currently available data constrain invisible ( or undetected ) @xmath23 decays . by performing fits to all public data from the lhc and the tevatron experiments , we have shown that the 95% cl limits for @xmath40 obtained depend very much upon the higgs sector model . assuming a higgs boson with sm couplings , @xmath202 is allowed at 95%  cl . allowing the @xmath18 , @xmath19 and @xmath1
reduced coupling factors ( that are defined as the coefficients multiplying the up - quark , down - quark and vector boson couplings relative to the sm values ) to deviate from unity , much larger invisible decay rates are possible . in the most theoretically motivated case where @xmath90 , @xmath203 and @xmath94 the latter being required for any two - doublet or two - doublet plus singlets model  we find @xmath204 at 95%  cl .
limits on invisible ( or yet undetected ) higgs decays also depend strongly on whether or not there are bsm particles that provide extra loop contributions to the @xmath24 and @xmath8 couplings of the @xmath23 . in the simplest case of an sm - like higgs with bsm loop contributions to its @xmath24 and @xmath8 couplings , the invisible decay rate can go up to about 60% at 95%  cl .    in the absence of invisible decays ,
the best fits to the lhc data suggest significant deviations of @xmath18 and/or @xmath205 from unity .
we have shown that including the additional possibility of invisible / undetected @xmath23 decays makes even larger deviations accompanied by large @xmath40 values consistent with lhc observations .
if @xmath40 really were as large as the @xmath7 limits we derived , one might hope that direct detection of invisible and/or hard - to - detect @xmath23 decays would be possible .
estimates suggest that @xmath206 at @xmath153 will typically be required .
finally , we have also shown that if @xmath207 is due to @xmath23 decays to a pair of dm particles , there are significant constraints on the size of @xmath40 from the non - observation of spin - independent dm scattering on nucleons , the most important such limits currently being those from the xenon100 experiment .
these constraints are much stronger for scalar dm than for majorana or dirac fermions .
overall , our results suggest a continued competition between limits on @xmath184 and those on @xmath40 as direct detection experiments achieve improved sensitivity and increasingly accurate measurements of the properties of the @xmath23 become available with future lhc running .    in short , precision measurements of the properties of the @xmath23 could well continue to provide the strongest constraints on a number of types of bsm physics , including the existence of light ( mass @xmath208 ) weakly interacting massive or hard - to - detect bsm particles .
while this paper was being refereed , atlas and cms presented major updates of their higgs results based on @xmath209  fb@xmath113 of data in most channels . taking these new results into account , we find @xmath210 at 95%  cl for a higgs boson with sm couplings , and @xmath211 at 95%  cl when @xmath90 , @xmath203 and @xmath94 are left free to vary  @xcite .
gb thanks a. pukhov for useful discussions .
this work was supported in part by us doe grant de - fg03 - 91er40674 and by in2p3 under contract pics fr
usa no .  5872 .
gb and sk acknowledge morever partial support from the french anr dmastrolhc .
ue acknowledges partial support from the french anr  lfv - cpv - lhc , anr  str - cosmo and the european union fp7 itn invisibles ( marie curie actions ,  pitn - ga-2011 - 289442 ) .
this work originated from discussions started at the aspen center for physics ( acp ) which is supported by the national science foundation grant no .
phy-1066293 ; gb , ue , jfg and sk thank the acp for hospitality and an inspiring working atmosphere .
gb , jfg and sk also thank the galileo galilei institute for theoretical physics ( ggi florence ) for hospitality and the infn for partial support .
finally , jfg acknowledges the hospitality of the kavli institute for theoretical physics which is supported by the national science foundation under grant no .
nsf phy11 - 25915 .
10 collaboration , g.  aad _ et al .
_ , `` observation of a new particle in the search for the standard model higgs boson with the atlas detector at the lhc , '' _ phys.lett . _ * b716 * ( 2012 ) 129 , http://www.arxiv.org/abs/1207.7214[1207.7214 ] .
collaboration , s.  chatrchyan _ et al .
_ , `` observation of a new boson at a mass of 125 gev with the cms experiment at the lhc , '' _ phys.lett .
_ * b716 * ( 2012 ) 3061 , http://www.arxiv.org/abs/1207.7235[1207.7235 ] .
, `` updated combination of cdf and d0 searches for standard model higgs boson production with up to 10.0 fb@xmath113 of data , '' http://www.arxiv.org/abs/1207.0449[1207.0449 ] .
g.  belanger , b.  dumont , u.  ellwanger , j.  gunion , and s.  kraml , `` higgs couplings at the end of 2012 , '' http://www.arxiv.org/abs/1212.5244[1212.5244 ] .
j.  r. espinosa , m.  muhlleitner , c.  grojean , and m.  trott , `` probing for invisible higgs decays with global fits , '' _ jhep _ * 1209 * ( 2012 ) 126 , http://www.arxiv.org/abs/1205.6790[1205.6790 ] .
d.  carmi , a.  falkowski , e.  kuflik , t.  volansky , and j.  zupan , `` higgs after the discovery : a status report , '' _ jhep _ * 1210 * ( 2012 ) 196 , http://www.arxiv.org/abs/1207.1718[1207.1718 ] .
p.  p. giardino , k.  kannike , m.  raidal , and a.  strumia , `` is the resonance at 125 gev the higgs boson ?
_ * b718 * ( 2012 ) 469474 , http://www.arxiv.org/abs/1207.1347[1207.1347 ] .
p. giardino , k.  kannike , m.  raidal , and a.  strumia , `` reconstructing higgs boson properties from the lhc and tevatron data , '' _ jhep _ * 1206 * ( 2012 ) 117 , http://www.arxiv.org/abs/1203.4254[1203.4254 ]
. n.  desai , b.  mukhopadhyaya , and s.  niyogi , `` constraints on invisible higgs decay in mssm in the light of diphoton rates from the lhc , '' http://www.arxiv.org/abs/1202.5190[1202.5190 ] .
j.  gunion , `` detecting an invisibly decaying higgs boson at a hadron supercollider , '' _ phys.rev.lett . _
* 72 * ( 1994 ) 199202 , http://www.arxiv.org/abs/hep-ph/9309216[hep-ph/9309216 ] .
d.  choudhury and d.  roy , `` signatures of an invisibly decaying higgs particle at lhc , '' _
_ * b322 * ( 1994 ) 368373 , http://www.arxiv.org/abs/hep-ph/9312347[hep-ph/9312347 ] .
o.  j. eboli and d.  zeppenfeld , `` observing an invisible higgs boson , ''
_ phys.lett .
_ * b495 * ( 2000 ) 147154 , http://www.arxiv.org/abs/hep-ph/0009158[hep-ph/0009158 ] .
r.  godbole , m.  guchait , k.  mazumdar , s.  moretti , and d.  roy , `` search for ` invisible ' higgs signals at lhc via associated production with gauge bosons , '' _ phys.lett .
_ * b571 * ( 2003 ) 184192 , http://www.arxiv.org/abs/hep-ph/0304137[hep-ph/0304137 ] .
h.  davoudiasl , t.  han , and h.  e. logan , `` discovering an invisibly decaying higgs at hadron colliders , '' _ phys.rev . _
* d71 * ( 2005 ) 115007 , http://www.arxiv.org/abs/hep-ph/0412269[hep-ph/0412269 ] .
r.  e.  shrock and m.  suzuki , `` invisible decays of higgs bosons , '' _
_  * b110 * ( 1982 ) 250 .
y.  bai , p.  draper and j.  shelton , `` measuring the invisible higgs width at the 7 and 8 tev lhc , '' _ jhep _ * 1207 * ( 2012 ) 192 [ arxiv:1112.4496 [ hep - ph ] ] .
a.  djouadi , a.  falkowski , y.  mambrini , and j.  quevillon , `` direct detection of higgs - portal dark matter at the lhc , '' http://www.arxiv.org/abs/1205.3169[1205.3169 ] .
d.  ghosh , r.  godbole , m.  guchait , k.  mohan , and d.  sengupta , `` looking for an invisible higgs signal at the lhc , '' http://www.arxiv.org/abs/1211.7015[1211.7015 ] . m.  almarashi and s.  moretti , `` reinforcing the no - lose theorem for nmssm higgs discovery at the lhc , '' _
_ * d84 * ( 2011 ) 035009 , http://www.arxiv.org/abs/1106.1599[1106.1599 ] . c.  englert , m.  spannowsky , and c.  wymant , ``
partially ( in)visible higgs decays at the lhc , '' _
_ * b718 * ( 2012 ) 538544 , http://www.arxiv.org/abs/1209.0494[1209.0494 ] .
h.  e.  logan and m.  -a .
roy , `` higgs couplings in a model with triplets , '' phys .
d * 82 * ( 2010 ) 115011 [ arxiv:1008.4869 [ hep - ph ] ] .
a.  falkowski , s.  rychkov , and a.  urbano , `` what if the higgs couplings to w and z bosons are larger than in the standard model ? , '' _ jhep _ * 1204 * ( 2012 ) 073 , http://www.arxiv.org/abs/1202.1532[1202.1532 ] .
d.  choudhury , r.  islam , a.  kundu and b.  mukhopadhyaya , `` anomalous higgs couplings as a window to new physics , '' arxiv:1212.4652 [ hep - ph ] .
collaboration , s.  chatrchyan _ et al .
_ , `` search for dark matter and large extra dimensions in monojet events in @xmath212 collisions at @xmath111 tev , '' _ jhep _ * 1209 * ( 2012 ) 094 , http://www.arxiv.org/abs/1206.5663[1206.5663 ] .
c.  petersson , a.  romagnoni , and r.  torre , `` higgs decay with monophoton + met signature from low scale supersymmetry breaking , '' _ jhep _ * 1210 * ( 2012 ) 016 , http://www.arxiv.org/abs/1203.4563[1203.4563 ] .
e.  dudas , c.  petersson , and p.  tziveloglou , `` low scale supersymmetry breaking and its lhc signatures , '' http://www.arxiv.org/abs/1211.5609[1211.5609 ] .
c.  p.  burgess , m.  pospelov and t.  ter veldhuis , `` the minimal model of nonbaryonic dark matter : a singlet scalar , '' nucl .
b * 619 * ( 2001 ) 709 [ hep - ph/0011335 ] .
s.  kanemura , s.  matsumoto , t.  nabeshima and n.  okada , `` can wimp dark matter overcome the nightmare scenario ?
, '' phys .
d * 82 * ( 2010 ) 055026 [ arxiv:1005.5651 [ hep - ph ] ] .
x.  -g .
he and j.  tandean , `` hidden higgs boson at the lhc and light dark matter searches , '' phys .
d * 84 * ( 2011 ) 075018 [ arxiv:1109.1277 [ hep - ph ] ] .
y.  mambrini , `` higgs searches and singlet scalar dark matter : combined constraints from xenon 100 and the lhc , '' phys .
d * 84 * ( 2011 ) 115017 [ arxiv:1108.0671 [ hep - ph ] ] .
p.  j.  fox , r.  harnik , j.  kopp and y.  tsai , `` missing energy signatures of dark matter at the lhc , '' phys .
d * 85 * ( 2012 ) 056011 [ arxiv:1109.4398 [ hep - ph ] ] .
a.  djouadi , o.  lebedev , y.  mambrini and j.  quevillon , `` implications of lhc searches for higgs  portal dark matter , '' phys .
b * 709 * ( 2012 ) 65 [ arxiv:1112.3299 [ hep - ph ] ] . p.  junnarkar and a.  walker - loud , `` the scalar strange content of the nucleon from lattice qcd , '' http://www.arxiv.org/abs/1301.1114[1301.1114 ] .
g.  belanger , f.  boudjema , a.  pukhov and a.  semenov , `` micromegas3.1 : a program for calculating dark matter observables , '' arxiv:1305.0237 [ hep - ph ] .
g.  belanger , f.  boudjema , a.  pukhov , and a.  semenov , `` dark matter direct detection rate in a generic model with micromegas 2.2 , '' _ comput.phys.commun . _
* 180 * ( 2009 ) 747767 , http://www.arxiv.org/abs/0803.2360[0803.2360 ] .
collaboration , e.  aprile _ et al .
_ , `` dark matter results from 225 live days of xenon100 data , '' _ phys.rev.lett .
_ * 109 * ( 2012 ) 181301 , http://www.arxiv.org/abs/1207.5988[1207.5988 ] .
g.  belanger , b.  dumont , u.  ellwanger , j.  gunion , and s.  kraml , in preparation .


there are numerous situations in which additive ( main effects ) models are insufficient for predicting an outcome of interest . in medical diagnosis ,
the co - occurrence of two symptoms may lead a doctor to be confident that a patient has a certain disease whereas the presence of either symptom without the other would provide only a moderate indication of that disease .
this situation corresponds to a positive ( i.e. , synergistic ) interaction between symptom variables . on the other hand ,
suppose both symptoms convey redundant information to the doctor about the patient so that knowing both provides no more information about the disease status than either one on its own .
this situation is again not additive , but this time there is a negative interaction between symptoms .
fitting regression models with interactions is challenging when one has even a moderate number , @xmath0 , of measured variables , since there are @xmath1 interactions of order @xmath2 .
for this paper , we focus on the case of pairwise ( @xmath3 ) interaction models , although the ideas we develop generalize naturally to higher - order interaction models .
we consider a regression model for an outcome variable @xmath4 and predictors @xmath5 , with pairwise interactions between these predictors .
in particular our model has the form @xmath6 where @xmath7 . regardless of whether the predictors are continuous or discrete
, we will refer to the additive part as the `` main effect '' terms and the quadratic part as the `` interaction '' terms .
our goal is to estimate @xmath8 and @xmath9 , where @xmath10 and @xmath11 .
the factor of one half before the interaction summation is a consequence of our notational decision to deal with a symmetric matrix @xmath12 of interactions rather than a vector of length @xmath13 .
we take @xmath11 throughout this paper because it simplifies notation , but everything carries over if we remove this restriction .
indeed , we provide this as an option in the ` hiernet ` ( pronounced `` hair net '' ) package .
we observe a training sample , @xmath14 , and our goal is to select a subset of the @xmath15 main effect and interaction variables that is predictive of the response , and to estimate the values for the nonzero parameters of the model .
it is a well - established practice among statisticians fitting ( [ eq : model ] ) to only allow an interaction into the model if the corresponding main effects are also in the model .
such restrictions are known under various names , including `` heredity , '' `` marginality , '' and being `` hierarchically well - formulated '' [ @xcite ] .
there are two types of restrictions , which we will call _ strong _ and _ weak hierarchy _ : @xmath16 some statisticians argue that models violating strong hierarchy are not sensible . for example , according to @xcite ,    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ `` [ t]here is usually no reason to postulate a special position for the origin , so that the linear terms must be included with the cross - term . '' _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _    to see that violating strong hierarchy amounts to `` postulating a special position for the origin , '' consider writing an interaction model as @xmath17 .
first of all , we would only take @xmath18 if we have special reason to believe that the regression surface must go through the origin .
likewise , taking @xmath19 but @xmath20 would only be appropriate if we actually believe that @xmath21 s effect on @xmath4 should only be present specifically when @xmath22 is nonzero . in most situations
, we do not think that the variable @xmath22 that we measured is any more special than @xmath23 . yet
if our model with @xmath22 violates strong hierarchy , then our model with @xmath23 ( for any @xmath24 ) is strongly hierarchical .
this argument suggests that violations to hierarchy occur in special situations whereas hierarchy is the default .
another argument in favor of hierarchy has to do with statistical power . in the words of @xcite :    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ `` [ l]arge component main effects are more likely to lead to appreciable interactions than small components .
also , the interactions corresponding to larger main effects may be in some sense of more practical importance . ''
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _    in other words , rather than looking at all possible interactions , it may be useful to focus our search on those interactions that have large main effects .
indeed , the method we propose in this paper makes direct use of this principle .    as a final argument for hierarchy
, it is useful to distinguish between two notions of sparsity , which we will call _ parameter sparsity _ and _ practical sparsity_. parameter sparsity is what most statisticians mean by `` sparsity '' : the number of nonzero coefficients in the model .
practical sparsity is what someone actually collecting data cares about : the number of variables one needs to _ measure _ to make predictions at a future time .
the hierarchy restriction favors models that `` reuse '' measured variables whereas a nonhierarchical model does not .
the top left panel of figure  [ fig : olive ] gives a small example where this difference is manifest .
in fact , a simple calculation shows that this difference can be quite substantial : we can have a hierarchical and a nonhierarchical interaction model with the same parameter sparsity but with the nonhierarchical method having a practical sparsity of @xmath25 whereas the hierarchical method s practical sparsity is just @xmath2 .
for the strong hierarchical lasso .
filled nodes correspond to nonzero main effects , and edges correspond to nonzero interactions . ]    while taking these arguments to the extreme leads to the use of strong hierarchy exclusively , we develop the case of weak hierarchy in parallel throughout this paper
. weak hierarchy , as the name suggests , can be thought of as a compromise between strong hierarchy and imposing no such structure and appears as a principle in certain statistical methods such as classification and regression trees [ @xcite ] and multivariate additive regression splines [ @xcite ] .
the lasso [ @xcite ] is a method that performs both model selection and estimation .
it penalizes the squared loss of the data with an @xmath26-norm penalty on the parameter vector .
this penalty has the property of producing estimates of the parameter vector that are sparse ( corresponding to model selection ) .
given a design matrix @xmath27 and response vector @xmath28 , the lasso is the solution to the convex optimization problem , @xmath29 where @xmath30 is the vector of ones .
the penalty parameter , @xmath31 , controls the relative importance of fitting to the training data ( sum - of - squares term ) and of sparsity ( @xmath26 penalty term ) .
a natural extension of the lasso to our interaction model  ( [ eq : model ] ) would be to take @xmath32 $ ] and @xmath33 , where the columns of @xmath34 correspond to elementwise products of the columns of @xmath35 .
we will refer to this method as the _ all - pairs lasso _ since it is simply the lasso applied to a data matrix which includes all pairs of interactions ( as well as all main effects ) .
it is common with the lasso to standardize the predictors so that they are on the same scale . in this paper , we standardize @xmath35 so that its columns have mean 0 and standard deviation  1 ; we then form @xmath36 from these standardized predictors and , finally , center the resulting columns of @xmath36 . by centering @xmath37 and @xmath38
, we may take @xmath39 .
the lasso s @xmath26 penalty is neutral to the pattern of sparsity , allowing any sparsity pattern to emerge .
the notions of strong and weak hierarchy introduced in section  [ sec : strong - weak - hier ] represent situations in which we want to exclude certain sparsity patterns .
there has been a growing literature focusing on methods that produce _ structured sparsity _ [ @xcite , @xcite , @xcite , @xcite , @xcite , @xcite ] .
these methods make use of the _ group lasso _ penalty ( and generalizations thereof ) which , given a predetermined grouping of the parameters , induces entire groups of parameters to be set to zero [ @xcite ] . given a set of groups of variables , @xmath40 , these methods generalize the @xmath26 penalty by @xmath41 where @xmath42 , @xmath43 is @xmath44 projected onto the coordinates in @xmath45 , and @xmath46 is a nonnegative weight .
hierarchical structured sparsity is obtained by choosing @xmath40 to have nested groups .
for example , @xcite consider the penalty @xmath47 likewise , the framework of @xcite if specialized to this paper s focus would lead to a penalty of the form @xmath48 for some @xmath49 and @xmath50 .
in fact , @xcite suggest a penalty for generalized additive models with interactions that reduces to ( [ eq : bach ] ) in the linear model case , with @xmath51 and @xmath52 independent of @xmath53 .      here
, we propose a lasso - like procedure that produces sparse estimates of @xmath54 and @xmath12 while satisfying the strong or weak hierarchy constraint .
in contrast to much of the structured sparsity literature which is based on group lasso penalties , our approach , presented in section  [ sec : our - proposed - method ] , involves adding a set of convex constraints to the lasso .
although we find this form of constraint more naturally interpretable , we show ( remark  [ re3 ] ) that this problem can be equivalently expressed in a form that relates it to penalties from the structured sparsity literature such as  ( [ eq : bach ] ) .
a key advantage of our specific choice of penalty structure is that it admits a simple interpretation of the effect of the hierarchy demand . unlike other hierarchical sparsity methods , which do not pay much attention to the particular choice of norms (
as long as @xmath42 ) , our formulation is carefully tailored to allow it to be related directly back to the lasso , permitting one to understand specifically how hierarchy alters the solution ( section  [ sec : effect - constraint ] ) .
this feature of our estimator gives it a transparency that exposes the effects ( both positive and negative ! ) of the hierarchy constraint .
furthermore , our characterization suggests that the demand for hierarchy is  analogous to the demand for sparsity  a form of `` regularization . ''
we develop an unbiased estimator of the degrees of freedom of our method ( section  [ sec : degrees - freedom ] ) and an interpretable upper bound on this quantity , which also points to hierarchy as regularization . in particular , we show that we do not `` spend '' in degrees of freedom for main effects that are forced into the model by the hierarchy constraint .
another difference from much of the structured sparsity literature , which aims to develop a broad treatment of structured and hierarchical sparsity methods , is that our focus is narrowed to the problem of interaction models .
our restricted scope allows us to address specifically the performance of such a tool to this important problem . in section
[ sec : related - work ] , we review previous work on the problem of hierarchical interaction model fitting and selection .
these methods fall into three categories : multi - step procedures , which are defined by an algorithm [ @xcite , @xcite , @xcite , @xcite , @xcite , @xcite , @xcite ] ; bayesian approaches , which specify the hierarchy requirement through a prior [ @xcite ] ; and , most related to this paper s proposal , regularized regression methods , which are defined by an optimization problem [ @xcite , @xcite , @xcite , @xcite , @xcite ] . in section  [ sec : empirical - study ] , we study via simulation the statistical implications of imposing hierarchy on an interactions - based estimator under various scenarios ( in both the lasso and stepwise frameworks ) . in section  [ sec : algorithmics ] we present an efficient algorithm for computing our estimator .
real data examples are used to illustrate a distinction we draw between `` parameter sparsity '' and `` practical sparsity '' and to discuss hierarchy s role in promoting the latter .
in section  [ sec : sparsity - lasso ] , we introduced the _ all - pairs lasso _ , which can be written as @xmath55 where @xmath56 and @xmath57 is the loss function , typically@xmath58 , but may also include a ridge penalty on the coefficients as discussed later or may be substituted for the binomial negative log - likelihood .
the one - half factors in front of terms involving @xmath12 are merely a consequence of the notational choice to represent @xmath12 as a symmetric matrix ( with @xmath11 for @xmath59 ) . in this paper
, we propose a modification of the all - pairs lasso that produces models that are guaranteed to be hierarchical .
as motivation for our proposal , consider building hierarchy into the optimization problem as a constraint , @xmath60 \\[-8pt ] \nonumber & & \qquad\mbox{s.t.}\qquad \theta= \theta^t , \vert\theta_j \vert_1\le|\beta_j| \qquad\mbox{for } j=1,\ldots , p,\end{aligned}\ ] ] where @xmath61 denotes the @xmath53th row ( and column , by symmetry ) of @xmath12 .
notice that if @xmath62 , then @xmath63 and @xmath64 and thus @xmath65 and @xmath66 . while the added constraints enforce strong hierarchy , they are not convex , which makes  ( [ eq : nonconvex ] ) undesirable as a method . in this paper
, we propose a straightforward convex relaxation of ( [ eq : nonconvex ] ) , which we call the _ strong hierarchical lasso _ , @xmath67 \\[-8pt ] \nonumber & & \qquad\mbox{s.t . }
\qquad\theta= \theta^t ,   \left.\begin{aligned } \vert\theta_j\vert_1&\le{\beta^+}_j+ { \beta^-}_j \\ { \beta^+}_j&\ge0 , { \beta^-}_j\ge0 \end{aligned } \right\}\mbox { for } j=1,\ldots , p , \nonumber\end{aligned}\ ] ] where we have replaced the optimization variable @xmath8 by two vectors @xmath68 . after solving the above problem , our fitted model is of the form @xmath69 . while we might informally think of @xmath70 and @xmath71 as positive and negative parts of a vector @xmath72 , that is , that @xmath73 , this is not actually the case since at a solution we can have both @xmath74 and @xmath75
indeed , if we were to add the constraints @xmath76 for @xmath59 to ( [ eq : hiernet ] ) , then these would be positive and negative parts and so @xmath77 , giving us precisely problem ( [ eq : nonconvex ] ) .
this observation establishes that ( [ eq : hiernet ] ) is a convex relaxation of  ( [ eq : nonconvex ] ) .
the hierarchy constraints can be seen as an embedding into our method of david cox s `` principle '' that `` large component main effects are more likely to lead to appreciable interactions than small components . ''
the constraint @xmath78 budgets the total amount of interactions involving variable @xmath79 according to the relative importance of @xmath79 as a main effect .
one additional advantage of the convex relaxation is that the constraint is less restrictive .
if the best fitting model would have @xmath80 large but @xmath81 only moderate , this can be accommodated by making @xmath82 and @xmath83 both large .
[ rem1 ] another possibility for the hierarchy constraint that we have considered is @xmath84 ; however , we have found that this can lead to an overabundance of interactions relative to main effects .
[ rem2 ] it is desirable to include in the loss function @xmath85 an _ elastic net _ term , @xmath86 , to ensure uniqueness of the solution [ @xcite ] .
we think of @xmath87 as a fixed tiny fraction of @xmath88 , such as @xmath89 , rather than as an additional tuning parameter .
such a modification does not complicate the algorithm , but simplifies the study of the estimator . in all numerical examples and in the ` hiernet ` package , we use this elastic net modification .
[ re3 ] we prove in section  2 of the supplementary materials [ @xcite ] that ( [ eq : hiernet ] ) may equivalently be written as @xmath90 this reparameterization of the problem shows its similarities to the group lasso based methods . in place of the more standard penalty @xmath91 of ( [ eq : bach ] ) , we use @xmath92 . in section [ sec : effect - constraint ] , we show that this unusual choice of penalty admits a particularly simple interpretation for the effect of imposing hierarchy .    in section  [ sec : strong - weak - hier ]
, we also introduced the notion of weak hierarchy . by simply removing the symmetry constraint on @xmath12
, we get what we call the _ weak hierarchical lasso _ , @xmath93 \\[-8pt ] \nonumber & & \qquad\mbox{s.t.}\qquad \left.\begin{aligned } \vert\theta_j \vert_1&\le{\beta^+}_j+{\beta^-}_j \\ { \beta^+}_j&\ge0 , { \beta^-}_j\ge0 \\ \end{aligned } \right\}\mbox { for } j=1,\ldots , p.\end{aligned}\ ] ] even though at a solution to this problem , @xmath94 is not symmetric , we should think of the interaction coefficient as @xmath95 since this is what multiplies the interaction term @xmath96 when computing @xmath97 .
[ rem4 ] we can build further on the connection between ( [ eq : bach ] ) and ( [ eq : hiernet ] ) discussed in remark  [ re3 ] .
our removal of the symmetry constraint in ( [ eq : weak - hiernet ] ) is analogous to the technique of duplicating columns of the design matrix used in the overlap group lasso [ @xcite ] .
a favorable property that distinguishes our method from previous approaches discussed in section  [ sec : related - work ] is the relative transparency of the role that the hierarchy constraint plays in our estimator .
this aspect is developed in section [ sec : effect - constraint ] .
although our primary focus in this paper is on the gaussian setting of ( [ eq : model ] ) , our proposal extends straightforwardly to other situations , such as the logistic regression setting in which the response is binary . in this case
, we simply have @xmath98 be the appropriate negative log - likelihood , @xmath99 , where @xmath100^{-1}$ ] . in section  3 of the supplementary materials [ @xcite ] ,
we show that solving this problem requires only a minor modification to our primary algorithm .
it should also be noted that our estimator ( and the algorithms developed to compute it ) is designed for both the @xmath101 and @xmath102 setting .    as a preliminary example , consider predicting whether a sample of olive oil comes from southern apulia based on measurements of the concentration of @xmath103 fatty acids [ @xcite ] .
the dataset consists of @xmath104 samples , and we average our results over 100 random equal - sized train - test splits .
we compare three methods : ( a ) a standard lasso with main effects only ( mel ) , ( b ) the all - pairs lasso ( apl ) , and ( c ) the strong hierarchical lasso ( hl ) .
the top left panel of figure  [ fig : olive ] shows an interesting difference between hl and apl .
we see that , on average , at a parameter sparsity level of five , the hl model uses four of the measured variables whereas apl uses six . using the hierarchical model to classify a future olive oil
, we only need to measure four rather than six of the fatty acids .
the top right panel of figure  [ fig : olive ] shows the predictive performance ( versus the practical sparsity ) of the three methods .
it appears that hl enjoys the `` best of both worlds , '' matching the good performance of mel for low practical sparsity levels ( since it tends to pick out the main effects first ) and the good performance of apl at high practical sparsity levels ( since it can incorporate predictive interactions ) .
finally , the bottom panel of the figure provides a visual display of a sequence of hl s solutions ( by varying @xmath88 ) .
nonzero main effects are shown as filled nodes , and edges indicate nonzero interactions .
since all edges are incident to filled nodes , we see that strong hierarchy holds .    in the next section , we present several properties of our estimator that shed light on the effect of adding the convex hierarchy constraint to the lasso . among these properties
is an unbiased estimate of the degrees of freedom of our estimator .
we view this degrees - of - freedom result as valuable primarily for the sake of understanding the effect of hierarchy .
while such an estimate could be used for parameter selection , we prefer cross validation to select @xmath88 since this is more directly tied to the goal of prediction .
a key advantage of formulating an estimator as a solution to a convex problem is that it can be completely characterized by a set of optimality conditions , known as the karush
tucker ( kkt ) conditions .
these conditions are useful for understanding the effect that the hierarchy constraint in ( [ eq : hiernet ] ) and ( [ eq : weak - hiernet ] ) has on our solutions . in this section
, we will study the simplest case , taking @xmath98 to be the quadratic loss function with no elastic net penalty .
we let @xmath105 denote partial residuals ( where @xmath106 denotes elementwise multiplication , @xmath107 the vector of fitted values and @xmath108 the @xmath53th predictor ) , and we assume that @xmath109 . for linear regression ,
the kkt conditions are known as the normal equations and can be written as @xmath110 the all - pairs lasso solution satisfies @xmath111}{\vert x_j*x_k\vert^2},\ ] ] where @xmath112 denotes the soft - thresholding operator defined by @xmath113 .
written this way , we see that the lasso is similar to linear regression , but all coefficients are shrunken toward 0 , with some coefficients ( those for which @xmath114 ) set to zero .
it is instructive to examine the corresponding statements for the strong and weak hierarchical lasso methods.=-1    [ prop : constraint ] the coefficients of the strong and weak hierarchical lassos with @xmath115 and taking @xmath98 to be the quadratic loss ( with no elastic net penalty ) satisfy :    * strong : @xmath116}{\vert x_j*x_k\vert^2};\end{aligned}\ ] ] * weak : @xmath117}{\vert x_j*x_k\vert^2}\end{aligned}\ ] ]    for some @xmath118 , @xmath59 with @xmath119 when @xmath120 ( and likewise for @xmath121 ) .
see section  1 of the supplementary materials [ @xcite ] .
the @xmath122 appearing in the above two properties are optimal dual variables corresponding to the @xmath53th hierarchy constraint for the strong and weak hierarchical lasso problems , respectively .
when @xmath123 , we have @xmath119 ( or @xmath124 ) by complementary slackness .
comparing these expressions to those of the all - pairs lasso gives insight into the effect of the constraint .
property [ prop : constraint ] reveals that the overall form of the all - pairs lasso and hierarchical lasso methods is identical .
the difference is that the hierarchy constraint leads to a reduction in the shrinkage of certain main effects and an increase in the shrinkage of certain interactions . in particular , we see that when the hierarchy constraints are loose at the solution , that is , @xmath120 , the weak hierarchical lasso s optimality conditions become identical to the all - pairs lasso ( since @xmath124 ) for all coefficients involving @xmath108 . for the strong hierarchical lasso ,
when both the @xmath53th and @xmath2th constraints are loose , the optimality conditions match those of the all - pairs lasso for the coefficients of @xmath108 , @xmath125 and @xmath126 .
the methods differ when constraints are active , that is , when @xmath127 , which allows @xmath128 ( or @xmath121 ) to be nonzero .
intuitively , this case corresponds to the situation in which hierarchy would not have held `` naturally '' ( i.e. , without the constraint ) , and the corresponding dual variable plays the role of reducing @xmath129 in @xmath26-norm and increasing @xmath130 until the constraint is satisfied . the way in which the weak and strong hierarchical lasso methods perform this shrinkage is different , but both are identical to the all - pairs lasso when all constraints are loose .      in section
[ sec : our - proposed - method ] , we showed that adding the constraint @xmath131 would guarantee that hierarchy holds .
however , we have not yet shown that the same is true of the convex relaxation s constraint , @xmath132 . in particular , while @xmath133 , we could still have @xmath134 .
this would correspond to a model in which @xmath135 is used in the model , but @xmath79 is not .
intuitively , we would expect that if @xmath74 , then @xmath136 is analogous to getting an exact zero in linear regression ( i.e. , a zero probability event ) . in this section
, we establish that this is in fact the case .
in particular , we study ( [ eq : hiernet ] ) and ( [ eq : weak - hiernet ] ) where @xmath98 includes an elastic net term .
the importance of this modification is that it ensures uniqueness , simplifying the analysis .
as noted in remark  [ rem2 ] , we think of @xmath137 as a small , fixed proportion of @xmath88 rather than as a separate tuning parameter .    [
prop : strong - hier ] suppose @xmath37 is absolutely continuous with respect to the lebesgue measure on @xmath138 . if @xmath139 solves ( [ eq : hiernet ] ) , where @xmath98 is the quadratic loss with an @xmath87 ridge penalty , then strong hierarchy holds with probability 1 , that is , @xmath140    see appendix  [ app : proof - hier ] .    to understand how dropping the symmetry constraint leads to the `` or '' statement required of weak hierarchy , note that @xmath135 is in the weak hierarchical lasso model if and only if @xmath141 .
this holds only if @xmath62 or @xmath142 .
[ prop : weak - hier ] suppose @xmath37 is absolutely continuous with respect to the lebesgue measure on @xmath138 .
if @xmath139 solves ( [ eq : weak - hiernet ] ) , where @xmath98 is the quadratic loss with an @xmath87 ridge penalty , then weak hierarchy holds with probability 1 , that is , @xmath143    see appendix  [ app : proof - hier ] .      in classical statistics ,
the degrees of freedom of a procedure refer to the dimension of the space over which its fitted values can vary .
it is useful in that it provides a measure of how much `` fitting '' the procedure is doing .
this notion can be generalized to adaptive procedures such as the lasso [ @xcite , @xcite , @xcite , @xcite ] .
see ( ryan ) @xcite for a thorough discussion .
if given data @xmath28 , a procedure @xmath144 produces fitted values @xmath145 , the degrees of freedom of the procedure @xmath144 is defined to be @xmath146    [ prop : df ] suppose @xmath147 .
an unbiased estimate of the degrees of freedom of the strong hierarchical lasso , with quadratic loss and no ridge penalty , is given by @xmath148 where @xmath149 with @xmath36 containing the interactions , and @xmath150 is a projection matrix which depends on the sign pattern of @xmath139 and on the set of hierarchy constraints that are tight .
see appendix  [ app : df ] .
figure  [ fig : df ] provides a numerical evaluation of how well @xmath151 estimates @xmath152 .
we fix @xmath153 and @xmath9 , and we generate @xmath154 monte carlo replicates @xmath155 . for each replicate
, we fit the strong hierarchical lasso along a grid of @xmath88 values to get @xmath156 and @xmath157 . from these values , we compute monte carlo estimates of @xmath152 from the definition in ( [ eq : df - def ] ) and of @xmath158 $ ] .
estimates @xmath152 .
monte carlo estimates of @xmath158 $ ] ( @xmath37-axis ) versus monte carlo estimates of @xmath152 ( @xmath159-axis ) for a sequence of @xmath88 values ( circular ) are shown .
one - standard - error bars are drawn and are hardly visible .
our bound on the unbiased estimate is plotted with diamonds . ]    while @xmath160 can be calculated from the data and is therefore useful as an unbiased way of calibrating the amount of fitting the strong hierarchical lasso is doing , this expression is difficult to interpret .
however , it turns out that we can _ bound _ @xmath160 by a quantity that does make more sense :    [ prop : df - bound ] let @xmath161 , @xmath162 and @xmath163 .
then , @xmath164 holds almost surely , where @xmath165 .
see appendix  [ app : df ] .
by contrast , for the all - pairs lasso in the case that @xmath166 and the design matrix is full rank , we have @xmath167 $ ] [ @xcite ] .
in other words , the strong hierarchical lasso does not `` pay '' ( in terms of fitting ) for those main effects , @xmath168 , that are forced into the model by the hierarchy constraint to accommodate a strong interaction .
notice that we do pay for a nonzero main effect if _ both _ @xmath169 and @xmath170 are nonzero .
this makes sense since the constraint could be satisfied with just one of these variables nonzero , but in this case it is advantageous to the fit to make both nonzero . in figure
[ fig : df ] , we find that this bound is in expectation visually indistinguishable from @xmath171 $ ] .
there has been considerable interest in fitting interaction models in statistics and related fields .
we focus here on an overview of methods that aim at forming predictive models that satisfy the hierarchical interactions restriction .
many statistics textbooks discuss a simple stepwise procedure in which one iteratively considers adding or removing the `` best '' variable ( whether it be main effect or interaction ) ; they add that one should only consider including an interaction if its main effects are in the model [ e.g. , see backward elimination in @xcite , section  6.1.3 ] . in doing so , they are enforcing the strong hierarchy restriction . such procedures
are ubiquitous [ @xcite , @xcite ] as are more recent versions [ @xcite , @xcite , @xcite , @xcite ] .
another approach is to perform model selection first without considering hierarchy and then to include any lower - order terms necessary to satisfy hierarchy as a post - processing step [ @xcite ] .
finally , @xcite and @xcite consider modifying the lars algorithm [ @xcite ] so that hierarchy is enforced .
another set of procedures for building hierarchical interaction models comes from a bayesian viewpoint .
@xcite adapts the _ stochastic search variable selection _ ( ssvs ) approach of @xcite to produce strong or weak hierarchical interaction models .
ssvs makes use of a hierarchical normal mixture model to perform variable selection in regression .
every variable has a latent binary variable indicating whether it is `` active . ''
conditional on this latent variable , each coefficient is a 0-mean normal with variance determined by the latent importance of the coefficient .
the original ssvs paper chooses a prior in which the importance of each variable is an independent bernoulli .
@xcite introduces dependence into the prior so that @xmath172 is important only if @xmath173 and/or @xmath174 is important as well .
@xcite formulate a nonconvex optimization problem to get sparse hierarchical interaction models .
they write @xmath175 , where @xmath54 are the main effect coefficients and then apply @xmath26 penalties on @xmath54 and @xmath176 .
notice that @xmath177 implies @xmath178 and @xmath179 .
the nonconvexity arises in writing @xmath172 as the product of optimization variables .
most similar to this paper s proposal is a series of methods which formulate convex optimization problems to give sparse hierarchical interaction models . @xcite
modify the nonnegative garrote [ @xcite ] by adding linear inequality constraints to enforce hierarchy . in this sense ,
our method can be seen as the adaption of their approach to the lasso .    finally ,
as discussed in section  [ sec : sparsity - lasso ] , another set of convex methods makes use of the group lasso penalty [ @xcite ] .
@xcite [ and , relatedly , @xcite ] describe composite absolute penalties ( cap ) , a very broad class of penalties that can achieve group and hierarchical sparsity . to achieve `` hierarchical selection ,
'' they put forward the principle that a penalty of the form @xmath180 , with @xmath181 , induces @xmath182 to be zero only when @xmath183 is zero as well .
for hierarchical interaction models , they suggest a penalty of the form @xmath184 $ ] .
this framework has been developed in the structured sparsity literature [ e.g. , @xcite ] .
@xcite introduce vanish , which uses this nested - group principle to achieve hierarchical sparsity in the context of nonlinear interactions .
their penalty in the setting of ( [ eq : model ] ) is @xmath185 $ ] .
as noted in remark  [ re3 ] , our proposal is closer to cap and vanish than it may first appear .
our problem can be rewritten to have a penalty of the form @xmath186.$ ] in this sense , the penalty is in the spirit of cap and related methods although it does not quite fall into the class of cap ( since ours involves a sum of norms of norms ) .
it is most similar to vanish in that it combines all of @xmath61 into the term involving @xmath173 .
our main interest in this section is to study the advantages and disadvantages of restricting one s interaction models to those that honor hierarchy . clearly , the effectiveness of such a strategy depends on the true model generating the data .
we take @xmath187 and @xmath188 ( 435 two - way interactions ) and consider four scenarios :    truth is hierarchical : @xmath189 ;    truth is anti - hierarchical : @xmath190 ;    truth only has interactions : @xmath191 for all @xmath53 ;    truth only has main effects : @xmath192 for all @xmath193 .    in cases ( i ) , ( ii ) , ( iv ) , we set 10 elements of @xmath54 to be nonzero ( with random sign ) , and , in cases ( i ) , ( ii ) , ( iii ) , we set 20 elements of the submatrix of @xmath10 to be nonzero . the signal - to - noise ratio ( snr ) for the main effects part of the signal is about 1.5 whereas the snr for the interactions part is about 1 .
we study the effectiveness of the hierarchy constraint in the context of both the lasso and forward stepwise regression .
forward stepwise regression refers to a greedy strategy for generating a sequence of linear regression models in which we start with an intercept - only model and then at each step add the variable that leads to the greatest decrease in the residual sum of squares .
we choose forward stepwise as a basis of comparison since it has a simple modification that we think may be the hierarchical interactions approach most commonly used by statisticians .
the modification is to restrict the set of interactions that could be added at a given step to only those between main effect variables currently in the model .
a backward stepwise version of this approach is suggested in @xcite .
we compare six methods , corresponding to each cell of the following table :    [ cols="<,^,^,^",options="header " , ]     we will refer to this in the proofs that follow .      including the elastic net penalty , @xmath194 , is equivalent to replacing @xmath38 and @xmath37 in ( [ eq : linear ] ) by @xmath195 suppose we solve ( [ eq : linear ] ) with the above design matrix . by lemma  [ lem : project ] ,
@xmath196 for some @xmath197 satisfying @xmath198>0\ ] ] for all @xmath199 .
let @xmath200 be the linear operator that selects the part of a vector corresponding to the variable @xmath201 .
now , @xmath202 implies that @xmath203 and @xmath204 .
we showed earlier that we can not have @xmath205 and @xmath206 .
this means that for any @xmath193 , there must be an @xmath207 for which @xmath208 corresponds to @xmath209 or @xmath210 .
thus , @xmath211 means that @xmath212 or @xmath213 for each @xmath193 .
this implies that @xmath214 and thus @xmath215 .
we show now that @xmath216 in terms of our above notation , this is @xmath217 where @xmath218 is the vector with all zeros except for @xmath219 .
let @xmath220 , and consider the set @xmath221^t\bigl(e_j^+-e_j^-\bigr)=0 , \\ & & \hspace*{32pt}\bigl[({\widetilde{x}}_\varepsilon p)^+\bigl(z-\bigl(p{\widetilde{x}}_\varepsilon^t \bigr)^+w\bigr)\bigr]^te_j^+>0 , \\ & & \hspace*{77pt}\bigl[({\widetilde{x}}_\varepsilon p)^+\bigl(z-\bigl(p{\widetilde{x}}_\varepsilon^t \bigr)^+w\bigr)\bigr]^te_j^->0\bigr\}.\end{aligned}\ ] ] in light of ( [ eq : sets ] ) , fixing @xmath222 automatically specifies @xmath223 .
the outer union is restricted to those subsets @xmath222 of @xmath224 elements that would have @xmath225 and @xmath226 . the event in ( [ eq : zeroprob ] )
is contained in @xmath227 since it corresponds to the case in which @xmath222 is @xmath228 .
we begin by showing that @xmath228 is in this restricted union with probability one .
we have already argued that at a solution we must have @xmath229 for all @xmath193 .
now , @xmath74 and @xmath75 together imply that @xmath230 since otherwise we could lower the objective by reducing @xmath169 and @xmath170 without leaving the feasible set .
therefore , it would be sufficient to show that @xmath231 .
we do so by observing that @xmath232 is a finite union of zero probability sets .
we begin by establishing that @xmath233 .
to do so , we write @xmath234 for some @xmath235 .
now , @xmath236 , so we can write @xmath237 , where @xmath238 and thus @xmath239 . since @xmath240
, it follows that @xmath241 .
write @xmath242 for @xmath243 , and observe that @xmath244 $ ] so that @xmath245 \\
& = & \varepsilon\bigl[u_1^tu_2-u_1^{\theta^+t}u_2^{\theta^-}-u_1^{\theta ^-t}u_2^{\theta^+ } \bigr ] \\ & = & -\varepsilon\biggl[0+\sum_{jk}\bigl[u_1^{\theta^+ } \bigr]_{jk}\bigl[u_2^{\theta ^-}\bigr]_{jk}+ \bigl[u_1^{\theta^-}\bigr]_{jk}\bigl[u_2^{\theta^+ } \bigr]_{jk}\biggr].\end{aligned}\ ] ] now , for each @xmath193 we must have @xmath246 since @xmath247 .
thus , for each @xmath193 , there is a r4 or r5 row in @xmath248 and thus @xmath249 implies that @xmath250_{jk}=0 $ ] or @xmath251_{jk}=0 $ ] for each @xmath193 and likewise @xmath252_{jk}=0 $ ] or @xmath253_{jk}=0 $ ] .
therefore , @xmath254 .
now , @xmath255 now , @xmath256 since @xmath257 and @xmath258 and @xmath259 has full column rank .
thus , @xmath260 .
this completes the first part of the proof .
next , we show that @xmath261 as long as @xmath262\times e_j^\pm>0 $ ] . now , since @xmath263 , the only row of @xmath264 that has @xmath265 is the r1 row ; but clearly @xmath266
. thus , @xmath267 and @xmath268 .
it follows that @xmath269    putting these two parts of the proof together establishes that@xmath270 .
thus , @xmath227 is a finite union of lebesgue measure 0 sets .
this shows that @xmath231 as long as @xmath37 is absolutely continuous with respect to the lebesgue measure on @xmath138 .
an argument nearly identical to that of the previous section establishes that @xmath271 with probability one .
thus , if @xmath134 and @xmath272 , then both @xmath273 and @xmath274 .
it follows then that @xmath275 .
this establishes weak hierarchy .
the fit in terms of the active set is given by @xmath276 where @xmath277 .
of course , @xmath278 , and we can solve the kkt conditions to get the rest of the optimal dual variables in terms of the active set @xmath279+c,\ ] ] where @xmath280 satisfies @xmath281+c\ge0.$ ]    note that @xmath282 and thus @xmath228 and @xmath283 depend on @xmath37 even though we do not write this explicitly . we will continue writing @xmath284 to mean specifically @xmath285 . for @xmath286 in a neighborhood of @xmath37 , we might guess that @xmath287 and @xmath288 , where @xmath289+c , \\ g\bigl(y'\bigr)_{{\mathcal{a}}({\hat\phi } ) } & = & 0.\end{aligned}\ ] ] to verify this guess , we need to check that the pair @xmath290 satisfies the optimality conditions at @xmath286 , @xmath291 first of all , @xmath292 holds since @xmath293 and @xmath294 .
likewise , @xmath295 .
now @xmath296 , so by continuity of @xmath297 , we have @xmath298 for all @xmath286 in a small enough neighborhood , @xmath299 , of @xmath37 .
this establishes that @xmath300 .
now @xmath301 , so complementary slackness holds . to see that the first optimality condition holds , we can simply plug @xmath302 into the left - hand side .
all that remains is to show that @xmath303 . if we knew that @xmath304 , then by continuity of @xmath305 we could argue that over a small enough neighborhood , @xmath306 , @xmath307 .
however , it could be the case that @xmath308 for some @xmath207 , that is , @xmath309 .
nonetheless , one can show that there is a set @xmath310 of measure 0 for which @xmath311 implies that @xmath312 and @xmath313 for all @xmath286 in a neighborhood of @xmath37 .
lemma 9 of ( ryan ) @xcite proves this result for a nearly identical situation .
the fit @xmath314 is a piecewise affine function of @xmath37 . using stein s formula for the degrees of freedom [ as described in ryan , @xcite ]
, we get that @xmath315=e\bigl[{\operatorname{tr}}\bigl\{({\widetilde{x}}p ) ( { \widetilde{x}}p)^+\bigr\ } \bigr]=e\bigl[\operatorname{rank}({\widetilde{x}}p)\bigr],\ ] ] where @xmath277 .
we bound this by an estimate that is more interpretable : @xmath316 .
clearly , r2r7 are linearly independent rows .
thus , the rank of @xmath317 is at least @xmath318 .
now , an r1 row is linearly independent of r2r8 precisely when @xmath319 has @xmath320 . to see this , note that if @xmath321 , then r1 is certainly linearly independent of r2r8 and likewise for @xmath322 ; however if @xmath323 , then @xmath324 for all @xmath325 , and therefore this row of r1 lies in the span of r3r7 .
thus , this means there are @xmath326 additional linearly independent rows .
finally , we consider r8 .
clearly , r8 lies in the span of r4r5 for @xmath324 since @xmath327 at a solution .
but if @xmath328 , then it is linearly independent of r1r8
. therefore , r8 adds @xmath329 to the rank where we have used that @xmath330 at a solution ( since @xmath331 ) and recalling that @xmath332 for the rows of r8 . in summary , we have shown that the row - rank is @xmath333 recalling that there are @xmath224 columns , we get that @xmath334
the lagrangian of ( [ eq : prox ] ) is given by @xmath335 where @xmath336 is the dual variable corresponding to the hierarchy constraints and @xmath337 are the dual variables corresponding to the nonnegativity constraints . for notational convenience , we have written @xmath338 for @xmath61 , and we have dropped the subscripts on @xmath339 .
the kkt conditions are @xmath340 where @xmath341 is a subgradient of the absolute value function evaluated at @xmath342 .
the three conditions involving @xmath337 implies that @xmath343_+$ ] .
the stationarity condition involving @xmath344 implies that @xmath345 .
now , define @xmath346_+-[\tilde\beta^-+t\alpha]_+.$ ] the remaining kkt conditions involve @xmath347 alone : @xmath348 observing that @xmath297 is nonincreasing in @xmath336 and piecewise linear suggests finding @xmath347 as done in algorithm  [ alg : onerow ] .
_ inputs : _ @xmath349 .    1 .
find @xmath350 define @xmath351_+-[\tilde\beta_j^-+t\alpha]_+.$ ] 1 .
if @xmath352 , take @xmath353 and go to step 2 .
form knot the set @xmath354 and let @xmath355 .
3 .   evaluate @xmath356 for @xmath357 .
4 .   if @xmath358 for some @xmath357 , take @xmath359 and go to step 2 .
find adjacent knots , @xmath360 , such that @xmath361 .
take @xmath362/(p_2-p_1).\ ] ] 2 .
return @xmath363 $ ] and @xmath364_+$ ] .
we thank will fithian , max grazier - gsell , brad klingenberg , balasubramanian narasimhan and ryan tibshirani for useful conversations and two referees and an associate editor for helpful comments .