the idea behind flow control in data networks is to regulate the source rates in order to prevent network overload , and provide fair allocation of resources . in recent years
, extensive research has been devoted to the problem of network utility maximization , with the objective of optimizing network resource allocation through a combination of source - based flow control , routing , and scheduling .
the common theme is to assign a utility , as a function of the source rate , to a flow specified by a source - destination pair , and formulate an optimization problem that maximizes the sum of utilities ( e.g. , see  @xcite ) .
the optimal network control policy is revealed as the algorithm that solves the utility maximization problem .
source - based flow control implicitly requires all sources to react properly to congestion signals such as packet loss or delay .
congestion - insensitive traffic , however , is pervasive in modern networks .
udp - based applications such as video streaming or voip are increasingly popular and do not respond to congestion .
moreover , greedy or malicious users can inject excessive data into the network to either boost self - performance or bring down high - profile websites . in these circumstances
, the network can be temporarily overloaded and congestion - aware applications , e.g. , tcp - based traffic , may be adversely affected or even starved . in this context , source - based flow control may not be adequate .
there are other scenarios in which optimizing the source rates of data flows on a per - flow basis is ineffective .
the internet nowadays is vulnerable to distributed denial of service ( ddos ) attacks  @xcite , in which an attacker creates a large number of flows with different source addresses to overwhelm prominent websites .
similarly , in a multi - player online game , thousands of users require continuous access to the game servers .
there are also occasions in which a large number of users seek access to a website that broadcasts live events  @xcite ; this is the so - called _ flash crowd _
phenomenon  @xcite . in these situations ,
source - based flow control is ineffective because each individual flow uses little bandwidth , but their aggregate traffic can lead to severe link congestion near the service provider , and may starve other users in the network . a flow control scheme that can optimize a utility assigned to a collection of flows as opposed to optimizing the sum of per - flow
utilities can be used to cope with such scenarios . moreover , in order to cope with uncooperative flows , it is necessary to relocate the flow control functionality from untrusted network hosts to more secure ones , such as the web servers that provide service at the receiver end .    in this paper , we develop such _ receiver - based flow control _ policies using tools from stochastic network optimization  @xcite .
our main contributions are three - fold .
first , we formulate a utility maximization problem that can assign utilities to an aggregate of flows , of which the usual per - flow - based utility maximization is a special case .
second , given an arbitrary arrival rate matrix ( possibly outside the network s stability region ) , we characterize the corresponding achievable throughput region in terms of queue overflow rates .
third , using a novel decomposition of the utility functions , we design a network control policy consisting of : ( i ) a set of flow controllers at the receivers ; ( ii ) packet dropping mechanism at internal nodes ; and ( iii ) back - pressure routing at intermediate nodes .
the receiver - based flow controllers adjust throughput by modifying the differential backlogs between the receivers and their neighboring nodes  a small ( or negative ) differential backlog is regarded as a _ push - back _ mechanism to slow down data delivery to the receiver . to deal with data that can not be delivered due to network overload , we design a threshold - based packet dropping mechanism that discards data whenever queues grow beyond certain thresholds .
surprisingly , we show that our threshold - based packet dropping scheme , without the use of flow control , is sufficient to maximize the weighted sum throughput . moreover , the combined flow control and packet dropping mechanism has the following properties : ( i ) it works with finite - size buffers .
( ii ) it is nearly utility - optimal ( throughput - optimal as a special case ) and the performance gap from the optimal utility goes to zero as buffer sizes increase .
( iii ) it does not require the knowledge of arrival rates and therefore is robust to time - varying arrival rates that can go far beyond the network s stability region .
in addition , our control policy can be implemented only in parts of the network that include the receivers , treating the rest of the network as exogenous data sources ( see fig .
[ fig:105 ] for an example ) , and thus might be an attractive flow control solution for web servers .    , @xmath0 , and @xmath1 on the right , where @xmath1 is the only receiver .
the rest of the network on the right may be controlled by another network operator or follow a different network control scheme.,width=336 ]    there has been a significant amount of research in the general area of stochastic network control .
utility - optimal policies that combine source - end flow control with back - pressure routing have been studied in  @xcite ( and references therein ) .
these policies optimize per - flow utilities and require infinite - capacity buffers .
however , they are not robust in the face of uncooperative users who may not adhere to the flow control scheme . a closely related problem to
that studied in this paper is that of characterizing the queue overflow rates in lossless networks in overload . in a single - commodity network ,
a back - pressure policy is shown to achieve the most balanced queue overflow rates  @xcite , and controlling queue growth rates using the max - weight policy is discussed in  @xcite .
the queue growth rates in networks under max - weight and @xmath2-fairness policies are analyzed in  @xcite .
we finally note that the importance of controlling an aggregate of data flows has been addressed in  @xcite , and rate - limiting mechanisms in front of a web server to achieve some notion of max - min fairness against ddos attacks have been proposed in  @xcite .
an outline of the paper is as follows .
the network model is given in section  [ sec:601 ] .
we formulate the utility maximization problem and characterize the achievable throughput region in terms of queue overflow rates in section  [ sec:602 ] .
section  [ sec:801 ] introduces a threshold - based packet dropping policy that maximizes the weighted sum throughput without the use of flow control .
section  [ sec:501 ] presents a receiver - based flow control and packet dropping policy that solves the general utility maximization problem .
simulation results that demonstrate the near - optimal performance of our policies are given in sections  [ sec:801 ] and  [ sec:501 ] .
we consider a network with nodes @xmath3 and directed links @xmath4 .
assume time is slotted . in every slot ,
packets randomly arrive at the network for service and are categorized into a collection @xmath5 of classes .
the definition of a data class is quite flexible except that we assume packets in a class @xmath6 have a shared destination  @xmath7 .
for example , each class can simply be a flow specified by a source - destination pair .
alternatively , computing - on - demand services in the cloud such as amazon ( elastic compute cloud ; ec2 ) or google ( app engine ) can assign business users to one class and residential users to another . media streaming applications may categorize users into classes according to different levels of subscription to the service provider . while classification of users / flows in various contexts
is a subject of significant importance , in this paper we assume for simplicity that the class to which a packet belongs can be ascertained from information contained in the packet ( e.g. , source / destination address , tag , priority field , etc . ) .
let @xmath8 be the number of exogenous class  @xmath9 packets arriving at node @xmath10 in slot @xmath11 , where @xmath12 is a finite constant ; let @xmath13 for all @xmath11 .
we assume @xmath14 are independent across classes @xmath9 and nodes @xmath15 , and are i.i.d . over slots with mean @xmath16 = { \lambda_{n}^{(c)}}$ ] .    in the network ,
packets are relayed toward the destinations via dynamic routing and link rate allocation decisions .
each link @xmath17 is used to transmits data from node @xmath10 to node @xmath18 and has a fixed capacity @xmath19 ( in units of packets / slot ) . under a given control policy ,
let @xmath20 be the service rate allocated to class  @xmath9 data over link @xmath21 in slot @xmath11 .
the service rates must satisfy the link capacity constraints @xmath22    at a node @xmath15 , class @xmath9 packets that arrive but not yet transmitted are stored in a queue ; we let @xmath23 be the backlog of class  @xmath9 packets at node @xmath10 at time @xmath11 .
we assume initially @xmath24 for all @xmath10 and @xmath9 .
destinations do not buffer packets and we have @xmath25 for all @xmath9 and @xmath11 . for
now , we assume every queue @xmath23 has an infinite - capacity buffer ; we show later that our control policy needs only finite buffers . to resolve potential network congestion due to traffic overload
, a queue @xmath23 , after transmitting data to neighboring nodes in a slot , discards @xmath26 packets from the remaining backlog at the end of the slot .
the drop rate @xmath26 is a function of the control policy to be described later and takes values in @xmath27 $ ] for some finite @xmath28 .
the queue @xmath23 evolves over slots according to @xmath29^{+ } \\ &
\quad+ { a_{n}^{(c)}}(t ) + \sum_{a } \mu_{an}^{(c)}(t ) , \quad \forall c,\ \forall n\neq d_{c } , \end{split}\ ] ] where @xmath30 .
this inequality is due to the fact that endogenous arrivals may be less than the allocated rate @xmath31 when neighboring nodes do not have sufficient packets to send .    for convenience ,
we define the maximum transmission rate into and out of a node by @xmath32 throughout the paper , we use the following assumption .    [ assu:601 ] we assume @xmath33 .    from
, the sum @xmath34 is the largest amount of data that can arrive at a node in a slot ; therefore it is an upper bound on the maximum queue overflow rate at any node .
assumption  [ assu:601 ] ensures that the maximum packet dropping rate @xmath28 is no less than the maximum queue overflow rate , so that we can always prevent the queues from blowing up .
we assign to each class @xmath35 a utility function @xmath36 . given an ( unknown ) arrival rate matrix @xmath37 , let @xmath38 be the set of all achievable throughput vectors @xmath39 , where @xmath40 is the aggregate throughput of class @xmath9 received by the destination @xmath7 .
note that @xmath38 is a function of  @xmath41 .
we seek to design a control policy that solves the global utility maximization problem @xmath42 where the region @xmath38 is presented later in lemma  [ lem:201 ] .
we assume all @xmath36 functions are concave , increasing , and continuously differentiable .
for ease of exposition , we also assume the functions @xmath36 have bounded derivatives such that @xmath43 for all @xmath44 , where @xmath45 are finite constants .
that have unbounded derivatives as @xmath46 can be approximated by those with bounded derivatives .
for example , we may approximate the proportional fairness function @xmath47 by @xmath48 for some small @xmath49 . ]    as an example , consider the tree network in fig .
[ fig:102 ] that serves three classes of traffic destined for node @xmath1 .
class  @xmath50 data originates from two different sources @xmath51 and @xmath52 , and may represent the collection of users located in different parts of the network sending or requesting information from node @xmath1 . if class @xmath50 traffic is congestion - insensitive and overloads the network , without proper flow control class @xmath53 and @xmath54 will be starved due to the presence of class @xmath50 .
a utility maximization problem here is to solve @xmath55 where @xmath56 denotes the throughput of class @xmath50 data originating from @xmath51 ; @xmath57 , @xmath58 , and @xmath59 are defined similarly . note that this utility maximization  - is very different from , and generalizes , the traditional per - flow - based utility maximization .
the next lemma characterizes the set @xmath38 of all achievable throughput vectors in  .
[ lem:201 ] under i.i.d .
arrival processes with an arrival rate matrix @xmath60 , let @xmath38 be the closure of the set of all achievable throughput vectors @xmath39
. then @xmath61 if and only if there exist flow variables @xmath62 and queue overflow variables @xmath63 such that @xmath64 in other words , @xmath65    see appendix  [ appendix:201 ] .    in lemma  [ lem:201 ] ,
equation   is the flow conservation constraint stating that the total flow rate of a class into a node is equal to the flow rate out of the node plus the queue overflow rate .
equation   is the link capacity constraint .
the equality in   shows that the throughput of a class is equal to the sum of exogenous arrival rates less the queue overflow rates .
lemma  [ lem:201 ] is closely related to the network capacity region @xmath66 defined in terms of admissible arrival rates ( see definition  [ defn:101 ] ) ; their relationship is shown in the next corollary .
[ defn:101 ] the capacity region @xmath66 of the network is the set of all arrival rate matrices @xmath67 for which there exists nonnegative flow variables @xmath68 such that @xmath69    [ lem:501 ] an arrival rate matrix @xmath70 lies in the network capacity region @xmath66 if there exist flow variables @xmath71 such that flow conservation constraints   and link capacity constraints   hold with @xmath72 for all @xmath10 and @xmath9 .
corollary  [ lem:501 ] shows that @xmath70 is achievable if and only if there exists a control policy yielding zero queue overflow rates . in this case
the throughput of class @xmath9 is @xmath73 .
we remark that the solution to the utility maximization  - activates the linear constraints  ; thus the problem  - is equivalent to @xmath74 let @xmath75 be the optimal throughput vector that solves  - . if the arrival rate matrix @xmath70 is in the network capacity region @xmath66 , the optimal throughput is @xmath76 from corollary  [ lem:501 ] . otherwise , we have @xmath77 , where @xmath78 is the optimal queue overflow rate .      our control policy that solves  - has two main features .
first , we have a packet dropping mechanism discarding data from the network when queues build up .
an observation here is that , in order to optimize throughput and keep the network stable , we should drive the packet dropping rate to be equal to the optimal queue overflow rate .
second , we need a flow controller driving the throughput vector toward the utility - optimal point . to convert the control objective   into these two control features ,
we define , for each class @xmath35 , a utility function @xmath79 related to @xmath36 as @xmath80 where @xmath81 are control parameters to be decided later . using  , we have @xmath82.\ ] ] since @xmath83 are unknown constants , maximizing @xmath84 is the same as maximizing @xmath85.\ ] ] this equivalent objective   can be optimized by jointly maximizing the new utility @xmath86 at the receivers and minimizing the weighted queue overflow rates ( i.e. , the weighted packet dropping rates ) @xmath87 at each node @xmath10 .
optimizing the throughput at the receivers amounts to controlling the amount of data _ actually delivered_. this is difficult because the data available to the receivers at their upstream nodes is highly correlated with control decisions taken in the rest of the network . optimizing the packet dropping rates depends on the data available at each network node , which has similar difficulties . to get around these difficulties ,
we introduce auxiliary control variables @xmath88 and @xmath89 and consider the optimization problem @xmath90 \label{eq:901 } \\
\text{subject to } & \quad r_{c } = \nu_{c } , \quad \forall\ , c\in { \mathcal{c}}\label{eq:210 } , \\
& \quad { q_{n}^{(c)}}\leq { \varphi_{n}^{(c ) } } , \quad \forall\ , c\in { \mathcal{c}},\
n\neq d_{c}. \label{eq:211 } \\ & \quad \text{\eqref{eq:213}-\eqref{eq:605 } hold
. } \label{eq:902}\end{aligned}\ ] ] this is an equivalent problem to  - .
the constraints   and   can be enforced by stabilizing virtual queues that will appear in our control policy .
the new control variables @xmath91 and @xmath92 to be optimized can now be chosen freely unconstrained by past control actions in the network . introducing auxiliary variables and setting up virtual queues are at the heart of using lyapunov drift theory to solve network optimization problems .
for ease of exposition , we first consider the special case of maximizing the weighted sum throughput in the network .
for each class @xmath35 , we let @xmath93 for some @xmath94 .
we present a threshold - based packet dropping policy that , together with back - pressure routing , solves this problem .
surprisingly , flow control is not needed here .
this is because maximizing the weighted sum throughput is equivalent to minimizing the weighted packet dropping rate . indeed , choosing @xmath95 in  , we have @xmath96 for all classes @xmath9 , under which maximizing the equivalent objective   is the same as minimizing @xmath97 . in the next section , we will combine the threshold - based packet dropping policy with receiver - based flow control to solve the general utility maximization problem .      to optimize packet dropping rates , we set up a _ drop queue _ @xmath98 associated with each queue @xmath23 .
the packets that are dropped from @xmath23 in a slot , denoted by @xmath99 , are first stored in @xmath98 for eventual deletion . from
, we have @xmath100.\ ] ] note that the quantity @xmath99 is the _ actual _ packets dropped from @xmath23 , which is strictly less than the allocated drop rate @xmath101 if queue @xmath23 does not have sufficient data .
packets are permanently deleted from @xmath98 at the rate of @xmath102 $ ] in slot @xmath11 .
the queue @xmath98 evolves according to @xmath103^{+}+ { \widetilde{d}_{n}^{(c)}}(t).\ ] ] assume initially @xmath104 for all @xmath10 and @xmath9 , where @xmath105 is a control parameter . to be finite .
our choice of @xmath106 avoids unnecessary packet dropping in the initial phase of the system . ]
if queue @xmath98 is stabilized , then minimizing the service rate of @xmath98 effectively minimizes the time average of dropped packets at @xmath23 .
we propose the following policy .    ' '' ''    overload resilient algorithm ( @xmath107 )    ' '' ''    _ parameter selection : _
choose @xmath108 for all classes @xmath35 , where @xmath109 .
choose a parameter @xmath105 .
_ backpressure routing : _ over each link @xmath110 , let @xmath111 be the subset of classes that have access to link @xmath112 . compute the differential backlog @xmath113 for each class @xmath114 , where @xmath115 at the receiver @xmath7 . define @xmath116 we allocate the service rates @xmath117 let @xmath118 for all classes @xmath119 .    _
packet dropping : _ at queue @xmath23 , allocate the packet dropping rate @xmath101 ( see  ) according to @xmath120 where @xmath121 is a constant chosen to satisfy assumption  [ assu:601 ] . at the drop queue @xmath98 ,
allocate its service rate @xmath122 according to @xmath123    _ queue update : _ update queues @xmath23 according to   and update queues @xmath98 according to  - in every slot .    ' '' ''    the packet dropping subroutine in this policy is threshold - based .
the @xmath107 policy uses local queueing information and does not require the knowledge of exogenous arrival rates .
it is notable that network overload is autonomously resolved by each node making local decisions of routing and packet dropping .
[ lem:402 ] for each class @xmath35 , define the constants @xmath124 in the @xmath107 policy , queues @xmath23 and @xmath98 are deterministically bounded by @xmath125 in addition , we have @xmath126 for all @xmath10 , @xmath9 , and @xmath11 .
see appendix  [ appendix:601 ] .
in lemma  [ lem:402 ] , the value of @xmath127 is the finite buffer size sufficient at queue @xmath23 .
the parameter @xmath128 controls when queue @xmath23 starts dropping packets .
indeed , due to @xmath129 , the @xmath107 policy discards packets from @xmath23 only if @xmath130 .
the quantity @xmath131 is a controllable threshold beyond which we say queue @xmath23 is overloaded and should start dropping packets . as we see next , the performance of the @xmath107 policy approaches optimality as the buffer sizes increase .
[ thm:501 ] define the limiting throughput of class @xmath9 as @xmath132,\ ] ] where @xmath133 denotes the class @xmath9 packets received by node @xmath7 over link @xmath134 .
the @xmath107 policy yields the limiting weighted sum throughput satisfying @xmath135 where @xmath75 is the optimal throughput vector that solves  - under the linear objective function @xmath136 , @xmath105 is a control parameter , and @xmath0 is a finite constant defined as @xmath137,\ ] ] where @xmath138 denotes the cardinality of a set @xmath139 .
we omit the proof of theorem  [ thm:501 ] because it is similar to that of theorem  [ thm:502 ] presented later in the general case of utility maximization . from  , the @xmath107 policy yields near - optimal performance by choosing the parameter @xmath128 sufficiently large .
correspondingly , a large @xmath128 implies a large buffer size of @xmath140 .
as shown in corollary  [ lem:501 ] , if the arrival rate matrix @xmath70 lies in the network capacity region @xmath66 , then the optimal throughput for class @xmath9 is @xmath76 and   reduces to @xmath141 that we can choose @xmath128 arbitrarily large leads to the next corollary .
[ cor:601 ] the @xmath107 policy is ( close to ) throughput optimal .
we conduct simulations for the @xmath107 policy in the network shown in fig .
[ fig:101 ] .        the directed links @xmath142 and @xmath143 have the capacity of @xmath50 packet / slot .
there are three classes of traffic to be served ; for example , class 1 data arrives at node @xmath0 and is destined for node @xmath52 .
classes @xmath50 and @xmath53 compete for service over @xmath143 ; classes @xmath53 and @xmath54 compete for service over @xmath142 . each simulation below is run over @xmath144 slots .      in each class , we assume a bernoulli arrival process whereby @xmath145 packets arrive to the network in a slot with probability @xmath146 , and no packets arrive otherwise .
the arrival rate of each class is 2 packets / slot , which clearly overloads the network .
let @xmath40 be the throughput of class @xmath9 .
consider the objective of maximizing the weighted sum throughput @xmath147 ; the weights are rewards obtained by serving a packet in a class .
the optimal solution is : ( i ) always serve class @xmath50 at node @xmath0 because it yields better rewards than serving class @xmath53 .
( ii ) always serve class @xmath54 at node @xmath51although class @xmath53 has better rewards than class @xmath54 , it does not make sense to serve class @xmath53 at @xmath51 only to be dropped later at @xmath0 .
the optimal throughput vector is therefore @xmath148 .
consider another objective of maximizing @xmath149 . here , class @xmath53 has a reward that is better than the sum of rewards of the other two classes .
thus both nodes @xmath51 and @xmath0 should always serve class @xmath53 ; the optimal throughput vector is @xmath150 .
table  [ table:601 ] shows the near - optimal performance of the @xmath107 policy in both cases as @xmath128 increases .
we show that the @xmath107 policy is robust to time - varying arrival rates .
suppose class @xmath50 and @xmath54 have a fixed arrival rate of @xmath151 packets / slot .
the arrival rate of class @xmath53 is @xmath53 packets / slot in the interval @xmath152 and is @xmath146 packets / slot elsewhere .
we consider the objective of maximizing @xmath149 .
the network is temporarily overloaded in the interval @xmath153 ; the optimal time - average throughput in @xmath153 is @xmath150 as explained in the above case .
the network is underloaded in the interval @xmath154 , in which the optimal throughput is @xmath155 .
we use the following parameters here : @xmath156 , @xmath157 , @xmath158 , and @xmath159 .
table  [ table:603 ] shows the near - optimal throughput performance of the @xmath107 policy .
figure  [ fig:103 ] shows the sample paths of the queue processes @xmath160 , @xmath161 , @xmath162 , and @xmath163 in the simulation .
clearly the queues suddenly build up when the network enters the overload interval @xmath153 , but the backlogs are kept close to the upper bound @xmath140 without blowing up .
.the throughput performance of the @xmath107 policy under time - varying arrival rates . [ cols= "
< , < , < " , ]
in this paper we develop a receiver - based flow control and an in - network packet dropping strategy to cope with network overload .
our scheme is robust to uncooperative users who do not employ source - based flow control and malicious users that intentionally overload the network .
a novel feature of our scheme is a receiver - based backpressure / push - back mechanism that regulates data flows at the granularity of traffic classes , where packets can be classified based on aggregates of data flows .
this is in contrast to source - based schemes that can only differentiate between source - destination pairs .
we show that when the receiver - based flow control scheme is combined with a threshold - based packet dropping policy at internal network nodes , optimal utility can be achieved .
10 [ 1]#1 url@samestyle [ 2]#2 [ 2]l@#1=l@#1#2    m.  chiang , s.  h. low , a.  r. calderbank , and j.  c. doyle , `` layering as optimization decomposition : a mathematical theory of network architectures , '' _ proc .
ieee _ , vol .
95 , no .  1 ,
255312 , jan . 2007 .
kelly , a.  k. maulloo , and d.  k.  h. tan , `` rate control in communication networks : shadow prices , proportional fairness and stability , '' _ journal of the oper .
_ , vol .
49 , pp . 237252 , 1998 .
kelly , `` charging and rate control for elastic traffic , '' _
european trans . telecommunications _
, vol .  8 , pp . 3337 , 1997 .
s.  h. low and d.  e. lapsley , `` optimization flow control  i : basic algorithm and convergence , '' _ ieee / acm trans .
_ , vol .  7 , no .  6 , pp .
861874 , dec .
a.  l. stolyar , `` maximizing queueing network utility subject to stability : greedy primal - dual algorithm , '' _ queueing syst .
_ , vol .
50 , no .  4 , pp . 401457 , 2005 .
m.  j. neely , e.  modiano , and c .- p .
li , `` fairness and optimal stochastic control for heterogeneous networks , '' _ ieee / acm trans .
_ , vol .  16 , no .  2 , pp .
396409 , apr .
a.  eryilmaz and r.  srikant , `` joint congestion control , routing , and mac for stability and fairness in wireless networks , '' _ ieee j. sel .
areas commun . _ ,
24 , no .  8 , pp .
15141524 , aug .
 , `` fair resource allocation in wireless networks using queue - length - based scheduling and congestion control , '' _ ieee / acm trans .
_ , vol .  15 , no .  6 , pp .
13331344 , dec . 2007 .
x.  lin and n.  b. shroff , `` joint rate control and scheduling in multihop wireless networks , '' in _ ieee conf . decision and control ( cdc )
2004 , pp . 14841489 .
r.  k.  c. chang , `` defending against flooding - based distributed denial - of - service attacks : a tutorial , '' _ ieee commun . mag .
_ , vol .
40 , no .
4251 , oct . 2002 .
a.  srivastava , b.  b. gupta , a.  tyagi , a.  sharma , and a.  mishra , `` a recent survey on ddos attacks and defense mechanisms , '' in _ advances in parallel distributed computing _
communications in computer and information science.1em plus 0.5em minus 0.4emspringer berlin heidelberg , 2011 , vol .
570580 .
j.  borland , `` net video not yet ready for prime time , '' feb .
[ online ] .
available : http://news.cnet.com/2100-1033-221271.html    r.  mahajan , s.  m. bellovin , s.  floyd , j.  ioannidis , v.  paxson , and s.  shenker , `` controlling high bandwidth aggregates in the network , '' _ acm computer communication review _
32 , pp . 6273 , 2002 .
l.  georgiadis , m.  j. neely , and l.  tassiulas , `` resource allocation and cross - layer control in wireless networks , '' _ foundations and trends in networking _ ,
vol .  1 , no .  1 , 2006 .    m.  j. neely , _
stochastic network optimization with application to communication and queueing systems_.1em plus 0.5em minus 0.4em morgan & claypool , 2010 .
l.  georgiadis and l.  tassiulas , `` optimal overload response in sensor networks , '' _ ieee trans .
inf . theory _
52 , no .  6 , pp . 26842696 , jun
. 2006 .    c.  w. chan , m.  armony , and n.  bambos , `` fairness in overloaded parallel queues , '' 2011 , working paper .
d.  shah and d.  wischik , `` fluid models of congestion collapse in overloaded switched networks , '' _ queueing syst . _ ,
69 , pp . 121143 , 2011 .
r.  egorova , s.  borst , and b.  zwart , `` bandwidth - sharing in overloaded networks , '' in _ conf .
information science and systems ( ciss ) _ , princeton , nj , usa , mar .
2008 , pp . 3641 .
d.  k.  y. yau , j.  c.  s. lui , f.  liang , and y.  yam , `` defending against distributed denial - of - service attacks with max - min fair server - centric router throttles , '' _ ieee / acm trans .
_ , vol .  13 , no .  1 ,
2942 , feb .
2005 .    c.  w. tan , d .-
chiu , j.  c.  s. lui , and d.  k.  y. yau , `` a distributed throttling approach for handling high bandwidth aggregates , '' _ ieee trans .
parallel distrib .
_ , vol .  18 , no .  7 ,
pp . 983995 , jul .
s.  chen and q.  song , `` perimeter - based defense against high bandwidth ddos attacks , '' _ ieee trans .
parallel distrib .
_ , vol .  16 , no .  6 , pp . 526537 , jun . 2005
.    m.  j. neely , e.  modiano , and c.  e. rohrs , `` dynamic power allocation and routing for time - varying wireless networks , '' _ ieee j. sel .
areas commun . _ ,
23 , no .  1 ,
89103 , jan .
2005 .    m.  j. neely , `` super - fast delay tradeoffs for utility optimal fair scheduling in wireless networks , '' _ ieee j. sel . areas
_ , vol .
24 , no .  8 , pp . 14891501 ,
first we show  - are necessary conditions . given a control policy ,
let @xmath164 be the amount of class @xmath9 packets transmitted over link @xmath165 in the interval @xmath166 $ ] , and @xmath23 be the class @xmath9 packets queued at node @xmath10 at time @xmath11 . from the fact that the difference between incoming and outgoing packets at a node in @xmath166 $ ] is equal to the queue backlog at time @xmath11 , we have @xmath167 which holds for all nodes @xmath15 for each class @xmath168 . taking expectation and time average of
, we obtain @xmath169 } = \frac{{\mathbb{e}\left[{q_{n}^{(c)}}(t)\right]}}{t } + \frac{1}{t } { \mathbb{e}\left[\sum_{b } f_{nb}^{(c)}(t)\right]}.\ ] ] the link capacity constraints lead to @xmath170 } \leq { \mu_{\text{max}}}^{ab } , \quad\forall\ , ( a , b)\in { \mathcal{l}}.\ ] ] consider the sequences @xmath171/t$ ] and @xmath172/t$ ] indexed by @xmath11 in  . for each @xmath35 and @xmath173 , the sequence @xmath174/t , t\in{\mathbb{z}}^{+}\}$ ]
is bounded because the capacity of each link is bounded .
it follows from   that the sequence @xmath175/t , t\in{\mathbb{z}}^{+}\}$ ] is also bounded .
there is a subsequence @xmath176 such that limit points @xmath68 and @xmath177 exist and satisfy , as @xmath178 , @xmath179 } & \to f_{ab}^{(c ) } , \quad \forall\,c\in{\mathcal{c}},\ \forall\ , ( a , b)\in{\mathcal{l } } ,   \label{eq:601 } \\
\frac{1}{t_{k}}{\mathbb{e}\left[{q_{n}^{(c)}}(t_{k})\right ] } & \to { q_{n}^{(c ) } } , \quad \forall\,c\in{\mathcal{c}},\ \forall\,n\neq d_{c}. \label{eq:602}\end{aligned}\ ] ] applying  - to  - results in   and  .
define the throughput @xmath40 of class @xmath9 as @xmath180}.\ ] ] the inequality in   follows   and  . the equality in   results from summing   over @xmath15 .
to show the converse , it suffices to show that every interior point of @xmath38 is achievable .
let @xmath181 be an interior point of @xmath38 , i.e. , there exists @xmath182 such that @xmath183 .
there exist corresponding flow variables @xmath68 and @xmath177 such that @xmath184 in the flow system  - , by removing the subflows that contribute to queue overflows , we obtain new reduced flow variables @xmath185 and @xmath186 such that @xmath187 , @xmath188 , and @xmath189 define @xmath190 \frac{r_{c}}{\sum_{n } \hat{\lambda}_{n}^{(c)}-\epsilon/2}.\ ] ] from  , we have @xmath191 and @xmath192 .
it is not difficult to check that @xmath193 , @xmath194 , and @xmath195 .
combined with  - , we obtain @xmath196 where the first inequality is a strict one .
these inequalities show that the rate matrix @xmath197 is an interior point of the network capacity region @xmath66 in definition  [ defn:101 ] , and therefore is achievable by a control policy , such as the back - pressure policy  @xcite . therefore , the aggregate rate vector @xmath39 , where @xmath198 , is also achievable .
we prove lemma  [ lem:402 ] by induction .
first we show @xmath98 is deterministically bounded .
assume @xmath199 for some @xmath11 , which holds at @xmath200 because we let @xmath106 .
consider the two cases :    1 .   if @xmath201 , then from   we obtain @xmath202 where the second inequality uses  .
2 .   if @xmath203 , then the @xmath107 policy chooses @xmath204 at queue @xmath98 and we have @xmath205^{+ } + { d_{\text{max}}}\\ & \leq \max[{d_{n}^{(c)}}(t ) , { d_{\text{max } } } ] \leq { d_{\text{max}}}^{(c)},\end{aligned}\ ] ] where the last inequality uses the induction assumption .    from these two cases ,
we obtain @xmath206 .    similarly , we show @xmath23 is bounded .
assume @xmath207 for some @xmath11 , which holds at @xmath200 because we let @xmath208 .
consider the two cases :    1 .   if @xmath209 , then from   we get @xmath210 where the second inequality uses assumption  [ assu:601 ] .
2 .   if @xmath211 , the @xmath107 policy chooses @xmath212 at queue @xmath23 and we have @xmath213 where the third inequality uses induction assumption .
we conclude that @xmath214 .    finally , we show @xmath129 for all slots .
assume this is true at some time @xmath11 ; this holds when @xmath200 because we assume @xmath106 .
consider the two cases :    1 .   if @xmath215 , the @xmath107 policy chooses @xmath216 at queue @xmath98 and we have @xmath217 by induction assumption .
2 .   if @xmath218 , the @xmath107 policy chooses @xmath219 and we have @xmath220    the proof is complete .
[ [ appendix:501 ] ]    we construct a proper lypuanov drift inequality that leads to the @xmath221 policy .
let @xmath222 be the vector of all physical and virtual queues in the network . using the parameters @xmath223 and @xmath224 given in the policy
, we define the lyapunov function @xmath225^{2 } + \frac{1}{2}\sum_{c , n\neq d_{c } } \bigl[d_{n}^{(c)}(t)\bigr]^{2 } \\
+ \sum_{c\in{\mathcal{c } } } \big(e^{w{\big(z_{c}(t)-q\big ) } } + e^{w{\big(q - z_{c}(t)\big ) } } \big).\end{gathered}\ ] ] the last sum is a lyapunov function whose value grows exponentially when @xmath226 deviates in both directions from the constant @xmath224 .
this exponential - type lyapunov function is useful for both stabilizing @xmath226 and guaranteeing there is sufficient data in @xmath226 .
such exponential - type lyapunov functions are previously used in  @xcite to study the optimal utility - delay tradeoff in wireless networks .
we define the lyapunov drift @xmath227},\ ] ] where the expectation is with respect to all randomness in the system in slot @xmath11 .
define the indicator function @xmath228 if @xmath229 and @xmath230 otherwise ; define @xmath231 .
define @xmath232 .
the next lemma is proved in appendix  [ appendix:202 ] .
[ lem:401 ] the lyapunov drift @xmath233 under any control policy satisfies @xmath234 } \notag \\   & + v\sum_{nc}\theta_{c } \,{\mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right ] } \leq b + \sum_{nc } { q_{n}^{(c)}}(t ) { \lambda_{n}^{(c)}}\notag\\ & -\sum_{nc } { q_{n}^{(c)}}(t ) { \mathbb{e}\left[\sum_{b } \mu_{nb}^{(c)}(t ) + d_{n}^{(c)}(t ) - \sum_{a } \mu_{an}^{(c)}(t ) \mid h(t)\right ] } \notag \\ & - \sum_{nc } { d_{n}^{(c)}}(t ) { \mathbb{e}\left[{\varphi_{n}^{(c)}}(t)-{d_{n}^{(c)}}(t ) \mid h(t)\right ] } \notag \\ & -v \sum_{c } { \mathbb{e}\left[h_{c}\big(\nu_{c}(t)\big ) \mid h(t)\right ] } + v\sum_{nc } \theta_{c}\ , { \mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right ] } \notag \\ & - w \sum_{c }   { 1_{c}^{\text{r}}}(t ) e^{w(z_{c}(t)-q ) } \big({\mathbb{e}\left[\delta_{c}(t)\mid h(t)\right ] } - \frac{\epsilon}{2 } \big ) \notag \\ & + w \sum_{c } { 1_{c}^{\text{l}}}(t ) e^{w(q - z_{c}(t))}\big({\mathbb{e}\left [ \delta_{c}(t ) \mid h(t)\right ] } + \frac{\epsilon}{2 } \big),\label{eq:227}\end{aligned}\ ] ] where @xmath0 is a finite constant defined by @xmath235 + \\ & \quad 2 { \left|{\mathcal{n}}\right|}{\left|{\mathcal{c}}\right| } { d_{\text{max}}}^{2 } + { \left|{\mathcal{c}}\right|}\big [ w(2{\delta_{\text{max}}}+\epsilon ) + e^{w({\nu_{\text{max}}}+{\mu_{\text{max}}^{\text{in } } } ) } + e^{wq } \big ] . \end{split}\ ] ]    by isolating decisions variables in  , it is not difficult to verify that the @xmath221 policy observes the current network state @xmath236 and minimizes the right - hand side of   in every slot .
[ [ appendix:202 ] ]    we establish the lyapunov drift inequality in  .
applying to   the fact that @xmath237^{+ } = ( a - b - c)^{+}$ ] for nonnegative reals @xmath238 , @xmath239 , and @xmath9 , and using lemma  [ lem:101 ] in appendix  [ appendix:203 ] , we have @xmath240^{2 } - \big[q_{n}^{(c)}(t)\big]^{2 } \big ) \leq b_{q } - \\ & q_{n}^{(c)}(t)\ , \big ( \sum_{b } \mu_{nb}^{(c)}(t ) + d_{n}^{(c)}(t )   - a_{n}^{(c)}(t ) - \sum_{a } \mu_{an}^{(c)}(t ) \big ) , \end{split}\ ] ] where @xmath241 is a finite constant . from   and  , we get @xmath242^{+}+ { d_{n}^{(c)}}(t).\ ] ] similarly , we obtain for queue @xmath98 @xmath243^{2 } - \big[d_{n}^{(c)}(t)\big]^{2 } \big ) \\ \leq b_{d } - d_{n}^{(c)}(t)\ , \big ( \varphi_{n}^{(c)}(t ) - d_{n}^{(c)}(t ) \big),\end{gathered}\ ] ] where @xmath244 is a finite constant .
[ lem:202 ] given fixed constants @xmath245 and @xmath246 , we define @xmath247 then @xmath248,\end{gathered}\ ] ] @xmath249.\end{gathered}\ ] ]    define @xmath250 and @xmath251 ; we have @xmath252 and @xmath253 . equation   yields @xmath254 since @xmath255 , we have @xmath256 because the first term is bounded by the second term if @xmath257 , and is bounded by the third term otherwise .
multiplying both sides by @xmath258 yields @xmath259,\end{gathered}\ ] ] where the last term follows the taylor expansion of @xmath260 .
if we have @xmath261 then plugging   into   leads to  .
indeed , by definition of @xmath223 in   we get @xmath262 which uses @xmath263 .    also ,   leads to @xmath264 define the event @xmath265 as @xmath266 i.e. , all upstream nodes of @xmath7 have sufficient data to transmit , in which case we must have @xmath267 .
it follows @xmath268 where the second case follows that queue @xmath226 is always non - negative .
similar as  , from   we obtain @xmath269 , \end{split}\ ] ] in which we use the taylor expansion of @xmath270 . bounding the last term of   by   and multiplying the inequality by @xmath271
, we obtain  .
now , define the indicator function @xmath228 if @xmath229 and @xmath230 otherwise .
define @xmath231 .
then for any real number @xmath238 , @xmath272 similarly , we have @xmath273 using @xmath274 in   and @xmath275 in  , inequalities   and   lead to @xmath276,\end{gathered}\ ] ] @xmath277.\end{gathered}\ ] ]    finally , summing   and   over @xmath15 for each @xmath168 , summing   and   over @xmath168 , and taking conditional expectation , we have @xmath278 } \\ - \sum_{nc } { d_{n}^{(c)}}(t ) { \mathbb{e}\left[{\varphi_{n}^{(c)}}(t)-{d_{n}^{(c)}}(t ) \mid h(t)\right ] } \\ - w \sum_{c } { 1_{c}^{\text{r}}}(t ) e^{w(z_{c}(t)-q ) } \big({\mathbb{e}\left[\delta_{c}(t)\mid h(t)\right ] } - \frac{\epsilon}{2 } \big ) \\ + w \sum_{c } { 1_{c}^{\text{l}}}(t ) e^{w(q - z_{c}(t))}\big({\mathbb{e}\left[\delta_{c}(t)\mid h(t)\right ] } + \frac{\epsilon}{2 } \big),\end{gathered}\ ] ] where @xmath0 is a finite constant defined as @xmath279 .
\label{eq:1208}\end{aligned}\ ] ] the constants @xmath280 and @xmath281 are defined right after   and  , respectively .
adding to both sides of   @xmath282 } +   v\sum_{nc } \theta_{c}\ , { \mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right]}\ ] ] yields  .
the boundedness of the queues @xmath23 and @xmath98 follows the proof of lemma  [ lem:402 ] . for the queues @xmath226
, we again prove by induction .
suppose @xmath283 , which holds at @xmath200 because we let @xmath284 .
consider the two cases :    1 .   if @xmath285 , then by   @xmath286 2 .   if @xmath287 , then from   we obtain @xmath288 where the last inequality follows our choice of @xmath128 satisfying @xmath289 as mentioned in section  [ sec:1201 ] . as a result , in
we have @xmath290 where the inequality follows @xmath291 in  .
since all queues @xmath23 for @xmath15 are deterministically bounded by @xmath127 , we have @xmath292 for all nodes @xmath10 such that @xmath293 .
consequently , the @xmath221 policy does not transmit any class @xmath9 packets over the links @xmath294 and the virtual queue @xmath226 has no arrivals .
therefore @xmath295 .
we conclude that @xmath296 .
the proof is complete .
let the throughput vector @xmath75 and the flow variables @xmath297 be the optimal solution to the utility maximization  - .
we consider the stationary policy that observes the current network state @xmath236 and chooses in every slot :      the first part is a feasible allocation because the value of @xmath78 is at most @xmath302 , which is less than or equal to @xmath28 by assumption  [ assu:601 ] .
the second part is feasible because the link rates @xmath303 satisfy the link capacity constraints  .
the third part is feasible because we assume @xmath304 .    under this stationary policy , by using the equality in   and using
, we have @xmath305 in every slot . as a result , @xmath306 } &
= \mathbb{e}\big[\nu_{c}^{*}(t ) - \sum_{a } \mu_{ad_{c}}^{(c)*}(t )   \mid z_{c}(t ) \geq q\big ] \\ & = { \mathbb{e}\left[\nu_{c}^{*}(t ) - r_{c}^ { * } \mid z_{c}(t ) \geq q\right ] } = \frac{\epsilon}{2 } , \\ { \mathbb{e}\left[\delta_{c}^{*}(t)\mid z_{c}(t ) < q\right ] } & = 0.\end{aligned}\ ] ] the @xmath221 policy minimizes the right - hand side of   in every slot .
using the decisions in the stationary policy , we can upper bound   under the @xmath221 policy by @xmath307 } + v\sum_{nc } \theta_{c}\,{\mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right ] } \notag \\ & \leq b + { \left|{\mathcal{c}}\right| } \frac{w\epsilon}{2 } e^{wq}- v \sum_{c } h_{c}(\nu_{c}^{*}(t ) ) + v \sum_{nc } \theta_{c}\ , q_{n}^{(c ) * } \notag \\ & \ -\sum_{nc } { q_{n}^{(c)}}(t ) \big ( \sum_{b } f_{nb}^{(c ) * } + q_{n}^{(c ) * } - { \lambda_{n}^{(c)}}- \sum_{a } f_{an}^{(c)*}\big ) \notag \\ & = b_{1 } - v \sum_{c } h_{c}(\nu_{c}^{*}(t ) ) + v \sum_{nc}\theta_{c}\ , q_{n}^{(c ) * } , \label{eq:404}\end{aligned}\ ] ] where we define @xmath308 and @xmath0 is given in  . the last equality in   is because the variables @xmath309 and @xmath78 satisfy the flow conservation constraints  .
the utility functions @xmath36 are assumed to have bounded derivatives with @xmath43 for all @xmath44 . thus , the utility functions @xmath79 have bounded derivatives with @xmath310 for all @xmath44 . using @xmath311 , we have @xmath312 if @xmath301 then it is trivial . if @xmath313 then by mean - value theorem there exists @xmath314 such that @xmath315 using   in   yields @xmath307 } + v\sum_{nc } \theta_{c}\,{\mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right ] } \notag \\ & \leq b_{1 } - v \sum_{c } h_{c}(r_{c}^ { * } ) + v \sum_{nc } \theta_{c}\,q_{n}^{(c ) * } + \frac{v\epsilon}{2 } \sum_{c } ( m_{c}+\theta_{c } ) \notag \\ & = b_{1 } - v\sum_{c } g_{c}(r_{c}^ { * } ) + v\sum_{nc } \theta_{c}\,{\lambda_{n}^{(c)}}+ \frac{v\epsilon}{2 } \sum_{c } ( m_{c}+\theta_{c } ) ,   \label{eq:905}\end{aligned}\ ] ] where the last equality uses @xmath316 and @xmath317 in  .
the next lemma is useful .
the value of @xmath321 is the solution to the convex program  - . if @xmath229 , then we have @xmath322 and @xmath323 .
if @xmath324 , the problem  - reduces to @xmath325 for all @xmath326 , we have by mean - value theorem @xmath327 where the first inequality results from the choice of @xmath328 that yields @xmath329 for all @xmath330 . rearranging terms in  , for all @xmath326 we have @xmath331 therefore ,
the solution to the problem  - must satisfy @xmath332 . since @xmath333
, we conclude @xmath320 .
now , from lemma  [ lem:403 ] we have @xmath334 if @xmath335 then this is trivial . otherwise , if @xmath336 then by mean - value theorem we obtain for some @xmath337 that @xmath338 plugging   into   and rearranging terms yield @xmath339 } \notag \\ & \qquad+ v\sum_{nc } \theta_{c}\,{\mathbb{e}\left[{\varphi_{n}^{(c)}}(t)\mid h(t)\right ] } \notag \\ & \leq b_{1 } + \frac{3v \epsilon}{2 } \sum_{c } ( m_{c}+\theta_{c } ) - v\sum_{c } g_{c}(r_{c}^ { * } ) + v\sum_{nc } \theta_{c}\,{\lambda_{n}^{(c)}}. \label{eq:411}\end{aligned}\ ] ] define the time average @xmath340}.\ ] ] define @xmath341 , @xmath342 , and @xmath343 similarly . taking expectation and time average over @xmath344 in  , dividing by @xmath128 , rearranging terms , and applying jensen s inequality to the functions @xmath79 , we get @xmath345}}{vt } - \frac{3\epsilon}{2 } \sum_{c}(m_{c}+\theta_{c}).\end{gathered}\ ] ] adding and subtracting @xmath346 at the left - hand side of   and using the definition of @xmath79 yield @xmath347}}{vt } - \frac{3\epsilon}{2 } \sum_{c } ( m_{c}+\theta_{c}).\end{gathered}\ ] ] from   and   we have @xmath348 taking expectation and time average yields @xmath349/t , \label{eq:412 } \\
\overline{\widetilde{\nu}_{c}(t ) } = \sum_{a } \overline{\widetilde{\mu}_{ad_{c}}^{(c)}(t ) } + \mathbb{e}\big[z_{c}(0)-z_{c}(t)\big]/t . \label{eq:413}\end{gathered}\ ] ] now , by the law of flow conservation , the sum of exogenous arrival rates must be equal to the sum of delivered throughput , time averages of dropped packets , and queue growth rates .
in other words , we have for each class @xmath168 and for all slots @xmath11 @xmath350.\ ] ] combining  - yields @xmath351 } + \frac{1}{t } \sum_{n } { \mathbb{e}\left[{d_{n}^{(c)}}(t)+{q_{n}^{(c)}}(t)\right]}.\end{gathered}\ ] ]    finally , using the boundedness of queues @xmath23 , @xmath98 , and @xmath226 in lemma  [ lem:502 ] and the continuity of @xmath36 , we obtain from   and   that @xmath352 where the last equality of   uses the definition in  . taking a limit of   as @xmath353 and using   and
, we obtain @xmath354 where the constant @xmath355 , defined in  , is @xmath356 \\ & \quad + { \left|{\mathcal{c}}\right| } \big [ w ( 2{\delta_{\text{max}}}+ \epsilon )   +   e^{w({\nu_{\text{max}}}+ { \mu_{\text{max}}^{\text{in } } } ) } + \frac{w\epsilon}{2}e^{wq } + e^{wq}\big ] .
\end{split}\ ] ] the proof is complete .
[ lem:101 ] if a queue process @xmath357 satisfies @xmath358 where @xmath359 and @xmath360 are nonnegative bounded random variables with @xmath361 and @xmath362 , then there exists a positive constant @xmath0 such that @xmath363


adaptive filtering has attracted much research interest in both theoretical and applied issues for a long time@xcite . due to its good performance , easy implementation , and high robustness , least mean square ( lms ) algorithm @xcite has been widely used in various applications such as system identification , channel equalization , and echo cancelation .
the unknown systems to be identified are sparse in most physical scenarios , including the echo paths @xcite and digital tv transmission channels @xcite . in other words , there are only a small number of non - zero entries in the long impulse response . for such systems ,
the traditional lms has no particular gain since it never takes advantage of the prior sparsity knowledge . in recent years
, several new algorithms have been proposed based on lms to utilize the feature of sparsity .
m - max normalized lms ( mmax - nlms)@xcite and sequential partial update lms ( s - lms)@xcite decrease the computational cost and steady - state mean squared error ( mse ) by means of updating filter tap - weights selectively .
proportionate nlms ( pnlms ) and its improved version@xcite accelerate the convergence by setting the individual step size in proportional to the respective filter weights .
sparsity in adaptive filtering framework has been a long discussed topic @xcite .
inspired by the recently appeared sparse signal processing branch @xcite , especially compressive sampling ( or compressive sensing , cs)@xcite , a family of sparse system identification algorithms has been proposed based on @xmath1 norm constraint .
the basic idea of such algorithms is to exploit the characteristics of unknown impulse response and to exert sparsity constraint on the cost function of gradient descent .
specially , za - lms @xcite utilizes @xmath2 norm and draws the zero - point attraction to all tap - weights .
@xmath0-lms @xcite employs a non - convex approximation of @xmath0 norm and exerts respective attractions to zero and non - zero coefficients . the smoothed @xmath0 algorithm , which is also based on an approximation of @xmath0 norm , is proposed in @xcite and analyzed in @xcite .
besides lms variants , rls - based sparse algorithms @xcite and bayesian - based sparse algorithms @xcite have also been proposed .
it is necessary to conduct a theoretical analysis for @xmath0-lms algorithm .
numerical simulations demonstrate that the mentioned algorithm has rather good performance compared with several available sparse system identification algorithms @xcite , including both accelerating the convergence and decreasing the steady - state msd .
@xmath0-lms performs _ zero - point attraction _ to small adaptive taps and pulls them toward the origin , which consequently increases their convergence speed and decreases their steady - state bias . because most coefficients of a sparse system are zero , the overall identification performance is enhanced .
it is also found that the performance of @xmath0-lms is highly affected by the predefined parameters .
improper parameter setting could not only make the algorithm less efficient , but also yield steady - state misalignment even larger than the traditional algorithm .
the importance of such analysis should be further emphasized since adaptive filter framework and @xmath0-lms behave well in the solution of sparse signal recovery problem in compressive sensing@xcite .
compared with some convex relaxation methods and greedy pursuits @xcite , it was experimentally demonstrated that @xmath0-lms in adaptive filtering framework shows more robustness against noise , requires fewer measurements for perfect reconstruction , and recovers signal with less sparsity .
considering its importance as mentioned above , the steady - state performance and instantaneous behavior of @xmath0-lms are throughout analyzed in this work .
one contribution of this work is on steady - state performance analysis . because of the nonlinearity caused by the sparsity constraint in @xmath0-lms , the theoretical analysis is rather difficult . to tackle this problem and enable mathematical tractability ,
adaptive tap - weights are sorted into different categories and several assumptions besides the popular independence assumption are employed .
then , the stability condition on step size and steady - state misalignment are derived .
after that , the parameter selection rule for optimal steady - state performance is proposed .
finally , the steady - state msd gain is obtained theoretically of @xmath0-lms over the tradition algorithm , with the optimal parameter .
another contribution of this work is on instantaneous behavior analysis , which indicates the convergence rate of lms type algorithms and also arouses much attention @xcite . for lms and most of its linear variants ,
the convergence process can be obtained in the same derivation procedure as steady - state misalignment . however , this no longer holds for @xmath0-lms due to its nonlinearity . in a different way by utilizing the obtained steady - state msd as foundation , a taylor expansion is employed to get an approximated quantitative analysis of the convergence process . also , the convergence rates are compared between @xmath0-lms and standard lms .      in order to theoretically characterize the performance and
guide the selection of the optimal algorithm parameters , the mean square analysis has been conducted for standard lms and a lot of its variants . to the best of our knowledge ,
widrow for the first time proposed the lms algorithm in @xcite and studied its performance in @xcite .
later , horowitz and senne @xcite established the mathematical framework for mean square analysis via studying the weight vector covariance matrix and achieved the closed - form expression of mse , which was further simplified by feuer and weinstein @xcite .
the mean square performance of two variants , leaky lms and deficient length lms , were theoretically investigated in similar methodologies in @xcite and @xcite , respectively . recently , dabeer and masry @xcite put forward a new approach for performance analysis on lms without assuming a linear regression model . moreover , convergence behavior of transform - domain lms was studied in @xcite with second - order autoregressive process .
a summarized analysis was proposed in @xcite on a class of adaptive algorithms , which performs linear time - invariant operations on the instantaneous gradient vector and includes lms as the simplest case .
similarly , the analysis of normalized lms has also attracted much attention @xcite .
however , the methodologies mentioned above , which are effective in their respective context , could no longer be directly applied to the analysis of @xmath0-lms , considering its high non - linearity .
admittedly , nonlinearity is a long topic in adaptive filtering and not unique for @xmath0-lms itself .
researchers have delved into the analysis of many other lms - based nonlinear variants @xcite .
nevertheless , the nonlinearity of most above references comes from non - linear operations on _ the estimated error _ , rather than _ the adaptive tap - weights _ that @xmath0-lms mainly focuses on .
we have noticed that the mean square deviation analysis of za - lms has been conducted @xcite . however , this work is far different from the reference .
first of all , the literature did not consider the transient performance analysis while in this work the mean square behavior of both steady - state and convergence process are conducted .
moreover , considering @xmath0-lms is more sophisticated than za - lms , there are more parameters in @xmath0-lms than in za - lms , which enhances the algorithm performance but increases the difficulty of theoretical analysis .
last but not least , taking its parameters to a specific limit setting , @xmath0-lms becomes essentially the same as za - lms , which can apply the theoretical results of this work directly .
a preliminary version of this work has been presented in conference@xcite , including the convergence condition , derivation of steady - state msd , and an expression of the optimal parameter selection .
this work provides not only a detailed derivation for steady - state results , but also the mean square convergence analysis .
moreover , both the steady - state msd and the parameter selection rule are further simplified and available for analysis .
finally , more simulations are performed to validate the results and more discussions are conducted .
this paper is organized as follows . in section [ section :
review ] , a brief review of @xmath0-lms and za - lms is presented . then in section [ section : preliminaries ] , a few assumptions are introduced which are reasonable in a wide range of situations . based on these assumptions , section [ section : analysis ] proposes the mean square analysis .
numerical experiments are performed to demonstrate the theoretical derivation in section [ section : simulation ] and the conclusion is drawn in section [ section : conclusion ] .
the unknown coefficients and input signal at time instant @xmath3 are denoted by @xmath4^{\rm t}$ ] and @xmath5^{\rm t}$ ] , respectively , where @xmath6 is the filter length .
the observed output signal is @xmath7 where @xmath8 denotes the additive noise .
the estimated error between the output of unknown system and of the adaptive filter is @xmath9 where @xmath10^{\rm t}$ ] denotes the adaptive filter tap - weights .    in order to take the sparsity of the unknown coefficients into account , @xmath0-lms @xcite inserts an @xmath0 norm penalty into the cost function of standard lms .
the new cost function is @xmath11 where @xmath12 is a factor to balance the estimation error and the new penalty . due to the np hardness of @xmath0 norm optimization , a continuous function
is usually employed to approximate @xmath0 norm .
taking the popular approximation@xcite and making use of the first order taylor expansion , the recursion of @xmath0-lms is @xmath13 where @xmath14^{\rm t}$ ] and @xmath15 the last item in ( [ w_update ] ) is called _ zero - point attraction _ @xcite , @xcite , because it reduces the distance between @xmath16 and the origin when @xmath17 is small . according to ( [ l_0gradapprx ] ) and fig . [ fig : g_l0l1](a ) , obviously such attraction is non - linear and exerts varied affects on respective tap - weights .
this attraction is effective for the tap - weights in the interval @xmath18 , which is named _
attraction range_. in this region , the smaller @xmath17 is , the stronger attraction affects .
za - lms ( or sparse lms ) @xcite runs similar as @xmath0-lms .
the only difference is that the sparse penalty is changed to @xmath2 norm .
accordingly the zero - point attraction item of the former is defined as @xmath19 which is shown in fig .
[ fig : g_l0l1](b ) .
the recursion of za - lms is @xmath20 where @xmath21 is the parameter to control the strength of sparsity penalty . comparing the sub figures in fig .
[ fig : g_l0l1 ] , one can readily accept that @xmath22 exerts the various attraction to respective tap - weight , therefore it usually behaves better than @xmath23 . in the following analysis
, one will read that za - lms is a special case of @xmath0-lms and the result of this work can be easily extended to the case of za - lms .
as its improvement , reweighted za - lms ( rza - lms ) is also proposed in @xcite , which modifies the zero - point attraction term to @xmath24 where parameter @xmath25 controls the similarity between ( [ rzagradapprx ] ) and @xmath0 norm . please refer to fig .
[ fig : g_l0l1](c ) for better understanding the behavior of ( [ rzagradapprx ] ) . in section
v , both za - lms and rza - lms are simulated for the purpose of performance comparison .
-lms , ( b ) za - lms , ( c ) rza - lms.,width=307 ]      denote @xmath26 and @xmath27 as the steady - state msd and instantaneous msd after @xmath3 iterations for lms with zero - mean independent gaussian input , respectively .
the steady - state msd has the explicit expression @xcite of @xmath28 where @xmath29 and @xmath30 denote the power of input signal and additive noise , respectively , and @xmath31 is a constant defined by ( [ definedeltal ] ) in appendix [ const ] . for the convergence process , the explicit expression of instantaneous msd is implied in @xcite as @xmath32    the next one turns to za - lms , @xmath33 is used to denote the steady - state msd with white gaussian input .
reference @xcite reaches the conclusion that @xmath34 where @xmath35 is the solution to @xmath36 where @xmath37 denotes the number of non - zero unknown coefficients and @xmath38 is a constant defined by ( [ definedelta0 ] ) .
@xmath0-lms employs steepest descent recursively and is applicable to solving sparse system identification .
more generally , steepest ascent iterations are used in several algorithms in the field of sparse signal processing .
for example , researchers developed smoothed @xmath0 method @xcite for sparse decomposition , whose iteration includes a steepest ascent step and a projection step .
the first step is defined as @xmath39 where @xmath40 serves as step size , @xmath41^{\rm t}$ ] denotes the negative derivative to an approximated @xmath0 norm and takes the value @xmath42 after ( [ sl0_1 ] ) , a projection step is performed which maps @xmath43 to @xmath44 in the feasible set .
it can be seen that ( [ sl0_1 ] ) performs steepest ascent , which is similar to zero - point attraction in @xmath0-lms .
the iteration details and performance analysis of this algorithm are presented in @xcite and @xcite , respectively .
another algorithm , named iterative bayesian@xcite , also enjoys steepest ascent iteration as @xmath45 where @xmath40 denotes the step size and @xmath6 is a log posterior probability function .
analysis of this algorithm and its application to sparse component analysis in noisy scenario are presented in @xcite .
considering the nonlinearity of zero - point attraction , some preparations are made to simplify the mean square performance analysis .      because various affects are exerted in @xmath0-lms to the filter tap - weights according to their respective system coefficients
, it would be helpful to classify the unknown parameters , correspondingly , the filter tap - weights , into several categories and perform different analysis on each category separately . according to the attraction range and their strength ,
all system coefficients are classified into three categories as @xmath46 where @xmath47 . obviously , @xmath48 and @xmath49 . in the following text ,
derivations are firstly carried out for the three sets separately .
then a synthesis is taken to achieve the final results .
the following assumptions about the system and the predefined parameters are adopted to enable the formulation .
* input data @xmath50 is an _
_ zero - mean gaussian signal .
* tap - weights @xmath51 , input vector @xmath52 , and additive noise @xmath8 are mutually independent . * the parameter @xmath53 is so small that @xmath54 .    assumption ( i ) commonly holds while ( ii ) is the well - known independence assumption @xcite . assumption ( iii ) comes from the experimental observations , i.e. , a too large @xmath53 can cause much bias as well as large steady - state msd . therefore , in order to achieve better performance , @xmath53 should not be too large .
besides the above items , several regular patterns are supposed during the convergence and the steady state .    *
all tap - weights , @xmath51 , follow gaussian distribution .
* for @xmath55 , the tap - weight @xmath56 is assumed to have the same sign with the corresponding unknown coefficient . *
the adaptive weight @xmath56 is assumed out of the attraction range for @xmath57 , while in the attraction range elsewhere .
assumption ( iv ) is usually accepted for steady - state behavior analysis @xcite .
assumption ( v ) and ( vi ) are considered suitable in this work due to the following two aspects .
first , there are few taps violating these assumptions in a common scenario
. intuitively , only the non - zero taps with rather small absolute value may violate assumption ( v ) , while assumption ( vi ) may not hold for the taps close to the boundaries of the attraction range .
for other taps which make up the majority , these assumptions are usually reasonable , especially in high snr cases .
second , assumptions ( v ) and ( vi ) are proper for small steady - state msd , which is emphasized in this work .
the smaller steady - state msd is , the less tap - weights differ from unknown coefficients . therefore , it is more likely that they share the same sign , as well as on the same side of the attraction range .
based on the discussions above , those patterns are regarded suitable in steady state .
for the convergence process , due to fast convergence of lms - type algorithms , we may suppose that most taps will get close to the corresponding unknown coefficients very quickly , so these patterns are also employed in common scenarios .
as we will see later , some of the above assumptions can not always hold in whatever parameter setting and may restrict the applicability of some analysis below .
however , considering the difficulties of nonlinear algorithm performance analysis , these assumptions can significantly enable mathematical tractability and help obtain results shown to be precious in a large range of parameter setting .
thus , we consider these assumptions reasonable to be employed in this work .
based on the assumptions above , the mean and mean - square performances of @xmath0-lms are analyzed in this section .
define the misalignment vector as @xmath58 , combine ( [ expout ] ) , ( [ esterror ] ) , and ( [ w_update ] ) , one has @xmath59 taking expectation and using the assumption ( ii ) , one derives @xmath60 where _ overline _ denotes expectation .    * for @xmath57 , utilizing assumption ( vi ) , one has @xmath61 . * for @xmath62 , combining assumptions ( iii ) , ( v ) and ( vi ) , it can be derived that @xmath63 * for @xmath64 , noticing the fact that @xmath65 has the opposite sign with @xmath66 in interval @xmath67 and using assumptions ( iv ) and ( vi ) , it can be derived that @xmath61 .
thus , the bias in steady state is obtained @xmath68 in steady state , therefore , the tap - weights are unbiased for large coefficients and zero coefficients , while they are biased for small coefficients .
the misalignment depends on the predefined parameters as well as unknown coefficient @xmath69 itself .
the smaller the unknown coefficient is , the larger the bias becomes .
this tendency can be directly read from fig .
[ fig : g_l0l1](a ) . in the attraction range ,
the intensity of the zero - point attraction increases as tap - weights get more closing to zero , which causes heavy bias .
thus , the bias of small coefficients in steady state is the byproduct of the attraction , which accelerates the convergence rate and increases steady - state msd .
the condition on mean square convergence and steady - state msd are given by the following theorem .
[ theo1 ] in order to guarantee convergence , step - size @xmath40 should satisfy @xmath70 and the final mean square deviation of @xmath0-lms is @xmath71 where @xmath72 are defined in ( [ beta1])@xmath73([beta3 ] ) in appendix [ const ] , respectively .
the proof of theorem [ theo1 ] goes in appendix [ proof_steady_msd ] .
_ remark 1 _ : the steady - state msd of @xmath0-lms is composed of two parts : the first item in ( [ msd_expression ] ) is exactly the steady - state msd of standard lms ( [ msdlms ] ) , while the latter two items compose an additional part caused by zero - point attraction .
when @xmath53 equals zero , @xmath0-lms becomes the traditional lms , and correspondingly the additional part vanishes .
when the additional part is negative , @xmath0-lms has smaller steady - state msd and thus better steady - state performance over standard lms .
consequently , it can be deduced that the condition on @xmath53 to ensure @xmath0-lms outperforms lms in steady - state is @xmath74    _ remark 2 _ : according to theorem [ theo1 ] , the following corollary on parameter @xmath53 is derived .
[ coro : opt_kappa ] from the perspective of steady - state performance , the best choice for @xmath53 is @xmath75{\frac{\beta_1+\beta_2}{\beta_1-\beta_2}}-\sqrt[4]{\frac{\beta_1-\beta_2}{\beta_1+\beta_2}}\right ) , \label{kappabest}\ ] ] and the minimum steady - state msd is @xmath76    the proof of corollary @xmath77 is presented in appendix [ best_parameters ] .
please notice that in ( [ msd_opt ] ) , the first item is about standard lms and the second one is negative when @xmath78 is less than @xmath6 .
therefore , the minimum steady - state msd of @xmath0-lms is less than that of standard lms as long as the system is not totally non - sparse .
_ remark 3 _ : according to the theorem , it can be accepted that the steady - state msd is not only controlled by the predefined parameters , but also dependent on the unknown system in the following two aspects .
first , the sparsity of the system response , i.e. @xmath78 and @xmath6 , controls the steady - state msd .
second , significantly different from standard lms , the steady - state msd is relevant to the _ small coefficients _ of the system , considering the attracting strength appears in @xmath79 and @xmath80 .    here
we mainly discuss the effect of system sparsity as well as the distribution of coefficients on the minimum steady - state msd . based on the above results , the following corollary can be deduced .
[ coro : steady_q ] the minimum steady - state msd of ( [ msd_opt ] ) is monotonic increasing with respect to @xmath78 and attracting strength @xmath81 .    the validation of corollary @xmath82 is performed in appendix [ effect_sparsity ] .
the _ zero - point attractor _ is utilized in @xmath0-lms to draw tap - weights towards zero .
the more sparse the unknown system is , the less steady - state msd is .
similarly , small coefficients are biased in steady state and deteriorate the performance , which explains that steady - state msd is increasing with respect to @xmath81 .
_ remark 4 _ : according to ( [ convcondition ] ) , one knows that @xmath0-lms has the same convergence condition on step size as standard lms@xcite and za - lms@xcite .
consequently the effect of @xmath40 on steady - state performance is analyzed .
it is indicated in ( [ msdlms ] ) that the standard lms enhances steady - state performance by reducing step size@xcite .
@xmath0-lms has a similar trend . for the seek of simplicity and practicability , a sparse system of @xmath78 far less than @xmath6
is considered to demonstrate this property . utilizing ( [ convcondition ] ) in such scenario ,
the following corollary is derived .
[ coro : steady_mu ] for a sparse system which satisfies @xmath83 the minimum steady - state msd in ( [ msd_opt ] ) is further approximately simplified as @xmath84 where @xmath85 and @xmath86 are defined by ( [ eta6 ] ) in appendix [ const ] , and @xmath81 , defined by ( [ attractingstrength ] ) , denotes the attracting strength to the zero - point .
furthermore , the minimum steady - state msd increases with respect to the step size .
the proof of corollary [ coro : steady_mu ] is conducted in appendix [ effect_mu ] . due to the stochastic gradient descent and zero - point attraction , the tap - weights suffer oscillation , even in steady state , whose intensity is directly relevant to the step size .
the larger the step size , the more intense the vibration .
thus , the steady - state msd is monotonic increasing with respect to @xmath40 in the above scenario .
_ remark 5 _ : in the scenario where @xmath87 remains a constant while @xmath88 approaches to zero , it can be readily accepted that ( [ w_update ] ) becomes totally identical to ( [ w_update_l1 ] ) , therefore @xmath0-lms becomes za - lms in this limit setting of parameters . in appendix [ relationship_sparse_standard_lms ]
it is shown that the result ( [ shi_sparse_lms_msd ] ) for steady - state performance @xcite could be regarded as a particular case of theorem [ theo1 ] .
as @xmath88 approaches to zero in @xmath0-lms , the attraction range becomes infinity and all non - zero taps belong to small coefficients which are biased in steady state .
thus , za - lms has larger steady - state msd than @xmath0-lms , due to bias of all taps caused by uniform attraction intensity .
if @xmath53 is further chosen optimal , the optimal parameter for za - lms is given by @xmath89 ( notice that @xmath90 approaches @xmath91 as @xmath88 tending to zero , as makes @xmath92 finite ) , and the minimum steady - state msd of @xmath0-lms ( [ msd_opt ] ) converges to that of za - lms . to better compare the three algorithms , the steady - state msds of lms , za - lms , and @xmath0-lms are listed in table [ table1 ] , where that of za - lms is rewritten and @xmath93 is defined in ( [ gamma_za ] ) in appendix [ relationship_sparse_standard_lms ] .
it can be accepted that the steady - state msds of both za - lms and @xmath0-lms are in the form of @xmath94 plus addition items , where @xmath94 denotes the steady - state msd of standard lms .
if the additional items are negative , za - lms and @xmath0-lms exceed lms in steady - state performance .
[ cols="^,^,^,^,^ " , ]     in the first experiment , the steady - state performance with respect to @xmath53 is considered . referring to fig .
[ fig : steady_msd_sweep_kappa_40db ] , the theoretical steady - state msd of @xmath0-lms is in good agreement with the experiment results when snr is @xmath95db . with the growth of @xmath53 from @xmath96 , the steady - state msd decreases at first , which means proper zero - point attraction is helpful for sufficiently reducing the amplitude of tap - weights in @xmath97 . on the other hand ,
larger @xmath53 results in more intensity of zero - point attraction item and increases the bias of small coefficients @xmath98 .
overlarge @xmath53 causes too much bias , thus deteriorates the overall performance . from ( [ kappabest ] )
, @xmath99 produces the minimized steady - state msd , which is marked with a square in fig .
[ fig : steady_msd_sweep_kappa_40db ] .
again , simulation result tallies with analytical value well . when snr is @xmath100db , referring to fig .
[ fig : steady_msd_sweep_kappa_20db ] , the theoretical result also predicts the trend of msd well .
however , since the assumptions ( v ) and ( vi ) do not hold well in low snr case , the theoretical result has perceptible deviation from the simulation result .    -lms ( with respect to different @xmath53 ) , where snr is @xmath95db and the solid square denotes @xmath101.,title="fig:",width=384 ] +    -lms ( with respect to different @xmath53 ) , where snr is @xmath100db and the solid square denotes @xmath101.,title="fig:",width=384 ] +    in the second experiment , the effect of parameter @xmath88 on steady - state performance is investigated .
please refer to fig .
[ fig : steady_msd_sweep_alpha_40db ] for results .
rza - lms is also tested for performance comparison , with its parameter @xmath21 chosen as optimal values which are obtained by experiments . for the sake of simplicity
, the parameter @xmath25 in ( [ rzagradapprx ] ) is set the same as @xmath88 .
simulation results confirm the validity of the theoretical analysis .
with very small @xmath88 , all tap - weights are attracted toward zero - point and the steady - state msd is nearly independent . as @xmath88 increases
, there are a number of taps fall in the attraction range while the others are out of it .
consequently , the total bias reduces .
besides , the results for za - lms are also considered in this experiment , with the optimal parameter @xmath21 proposed in _
remark 5_. it is shown that @xmath0-lms always yields superior steady - state performance than za - lms ; moreover , in scenario where @xmath88 approaches @xmath102 , the msd of @xmath0-lms tends to that of za - lms . in the parameter range of this experiment ,
@xmath0-lms shows better steady - state performance than rza - lms .    ) , and @xmath0-lms ( with respect to different @xmath88 ) , where @xmath25 equals @xmath88 .
parameters @xmath21 and @xmath53 are chosen as optimal for rza - lms and @xmath0-lms , respectively.,title="fig:",width=384 ] +    the third experiment studies the effect of non - zero coefficients number on steady - state deviation .
please refer to fig .
[ fig : steady_msd_sweep_q ] .
it is readily accepted that @xmath0-lms with optimal @xmath53 outperforms traditional lms in steady state .
the fewer the non - zero unknown coefficients are , the more effectively @xmath0-lms draws tap - weights towards zero .
therefore , the effectiveness of @xmath0-lms increases with the sparsity of the unknown system .
when @xmath78 exactly equals @xmath6 , its performance with optimal @xmath53 already attains that of standard lms , indicating that there is no room for performance enhancement of @xmath0-lms for a totally non - sparse system .
-lms ( with respect to different total non - zeros taps @xmath78 ) , where @xmath53 is chosen as optimal.,title="fig:",width=384 ] +    the fourth experiment is designed to investigate convergence process with respect to @xmath53 . also , the learning curve of the standard lms is simulated . when snr is @xmath95db , the results in fig .
[ fig : converegence_msd_sweep_kappa_40db ] demonstrate that our theoretical analysis of convergence process is generally in good accordance with simulation .
it can be observed that different @xmath53 results in differences in both steady - state msd and the convergence rate . due to more intense zero - attraction force ,
larger @xmath53 results in higher convergence rate ; but too large @xmath53 can have bad steady - state performance for too much bias of small coefficients .
moreover , @xmath0-lms outperforms standard lms in convergence rate for all parameters we run , and also surpasses it in steady - state performance when @xmath53 is not too large . when snr is 20db , fig .
[ fig : converegence_msd_sweep_kappa_20db ] also shows similar trend about how @xmath53 influences the convergence process ; however , since the low snr scenario breaks assumptions ( v ) and ( vi ) , the theoretical results and experimental results differ to some extent .    -lms ( with respect to different @xmath53 ) , where snr is @xmath95db.,title="fig:",width=384 ] +    -lms ( with respect to different @xmath53 ) , where snr is @xmath100db.,title="fig:",width=384 ] +    the fifth experiment demonstrates convergence process for various step sizes , with the comparison of lms and @xmath0-lms
. please refer to fig .
[ fig : converegence_msd_sweep_mu ] . similar to traditional lms ,
smaller step size yields slower convergence rate and less steady - state msd .
therefore , the choice of step size should seek a balance between convergence rate and steady - state performance . furthermore , the convergence rate of @xmath0-lms is faster than that of lms when their step sizes are identical .
-lms with respect to different step sizes @xmath40 , where @xmath53 is chosen as optimal for @xmath0-lms.,title="fig:",width=384 ] +
the comprehensive mean square performance analysis of @xmath0-lms algorithm is presented in this paper , including both steady - state and convergence process .
the adaptive filtering taps are firstly classified into three categories based on the zero - point attraction item , and then analyzed separately . with the help of some assumptions which are reasonable in a wide range ,
the steady - state msd is finally deduced and the convergence of instantaneous msd is approximately predicted . moreover , a parameter selection rule is put forward to minimize the steady - state msd and theoretically it is shown that @xmath0-lms with optimal parameters is superior than traditional lms for sparse system identification .
the all - round theoretical results are verified in a large range of parameter setting through numerical simulations .
in order to make the main body simple and focused , the explicit expressions of some constants used in derivations are listed here .
all through this work , four constants of @xmath103 are used to simplify the expressions .    to evaluate the zero - point attracting strength , with respect to the sparsity of the unknown system coefficients ,
two kinds of strengthes are defined as @xmath104 which are utilized everywhere in this work .
considering the attraction range , it can be readily accepted that these strengthes are only related to the small coefficients , other than the large ones and the zeros .    in lemma [ lemma ]
, @xmath105 is defined as @xmath106,\label{matrix_a}\ ] ] and @xmath107^{\rm t},\label{vector_b}\ ] ] where @xmath108 where @xmath109 is the solution to ( [ quadratic_equ ] ) .    in theorem
[ theo2 ] , the constants @xmath110 and @xmath111 are @xmath112    in corollary [ coro : opt_kappa ] , the constants @xmath72 are @xmath113    in appendix [ effect_sparsity ] and [ effect_mu ] , the constants @xmath114 are @xmath115
denote @xmath116 to be msd at iteration @xmath3 , and @xmath117 to be the second moment matrix of @xmath118 , respectively , @xmath119 substituting ( [ h_update ] ) into ( [ secondmoment ] ) , and expanding the term @xmath120 into three second moments using the gaussian moment factoring theorem @xcite , one knows @xmath121 using the fact that @xmath122 , one has @xmath123 consequently , the condition needed to ensure convergence is @xmath124 and ( [ convcondition ] ) is derived directly , which is the same as standard lms and similar with the conclusion in @xcite .    next the steady - state msd will be derived . using ( [ r_update ] ) and considering the @xmath125th diagonal element
, one knows @xmath126 to develop @xmath127 , one should first investigate two items , namely @xmath128 and @xmath129 in ( [ msd_comp_steadystate ] ) .
for @xmath57 , from assumption ( vi ) one knows @xmath130 , thus @xmath131 for small coefficients , considering assumptions ( v ) and ( vi ) , formula ( [ l_0gradapprx ] ) implies @xmath132 is a locally linear function with slope @xmath133 , which results in @xmath134 thus , it can be shown @xmath135 where @xmath136 is derived in ( [ h_steadystate_detail ] ) .    then turning to @xmath64
, it is readily known that @xmath137 in this case .
thus , from assumptions ( iv ) and ( vi ) , the following results can be derived from the property of gaussian distribution , @xmath138 combining assumption ( iii ) , ( [ h_steadystate_detail ] ) and ( [ ehgegg_lc])@xmath73([egg_sc ] ) , one can know the equivalency between ( [ msd_comp_steadystate ] ) and following equations for @xmath125 in @xmath139 , @xmath98 , and @xmath97 , respectively , @xmath140 where @xmath109 denotes @xmath141 for simplicity . summing ( [ msd_comp_steadystate_lc ] ) and ( [ msd_comp_steadystate_sc ] ) for all @xmath142 , and noticing that @xmath143 it could be derived that @xmath144 where @xmath81 is introduced in ( [ attractingstrength ] ) . combining ( [ msd_expression_old ] ) and ( [ msd_comp_steadystate_zc ] ) , it can be reached that @xmath109 is defined by the following equation @xmath145 finally , ( [ msd_expression ] ) is achieved after solving the quadratic equation above and a series of formula transformation on ( [ msd_expression_old ] ) .
thus , the proof of theorem [ theo1 ] is completed .      the vector @xmath187 in ( [ vector_b ] )
could be denoted as @xmath188 ,   \label{b_simp}\ ] ] where @xmath110 is defined in ( [ lambda3 ] ) and @xmath189 are constants .
take @xmath190-transform for ( [ d_omega_iter ] ) , it can be derived that @xmath191 = ( z\mathbf{i}-\mathbf{a})^{-1 } z \left[\!\!\ !
d_0\\ \omega_0\end{array}\!\!\!\right ] + ( z\mathbf{i}-\mathbf{a})^{-1}{\mathbf{b}(z)},\ ] ] where @xmath192 .
then combine the definition of @xmath193 in theorem [ theo2 ] and the above results , it is further derived @xmath194 where @xmath195 and @xmath196 are constants .
take the inverse @xmath190-transform and notice the definition of @xmath197 , it finally yields @xmath198 thus we have completed the proof of ( [ d_conv_closedform ] ) . by forcing the equivalence between ( [ d_conv_closedform ] ) and lemma [ lemma ] , the expression of @xmath111 could be solved as ( [ c3 ] ) .
by defining @xmath146 , ( [ msd_expression ] ) becomes @xmath147 where @xmath148 is defined as @xmath149 next we want to find @xmath150 which minimizes @xmath148 . forcing the derivative of @xmath148 with respect to @xmath66 to be zero
, it can be obtained that @xmath151 combining @xmath152 and substituting @xmath153 in ( [ msd_theta ] ) , corollary [ coro : opt_kappa ] can be finally achieved .      from ( [ msdlms ] ) , ( [ msd_opt ] ) , ( [ beta1 ] ) , ( [ beta2 ] ) , and ( [ eta0 ] )
, it can be obtained that @xmath154 note neither the @xmath26 defined in ( [ msdlms ] ) nor @xmath155 defined in ( [ eta0 ] ) is dependent on @xmath78 or @xmath156 , thus the focus of the proof is the denominator in ( [ msd_opt_eta ] ) . in the following
, we will analyze the two items in the denominator separately and obtain their monotonicity . the first item in the denominator
is @xmath157 from ( [ beta1_coro : steady_q ] ) , it has already shown that @xmath158 is increasing with respect to @xmath78 and @xmath156 .
next we consider the second item .
it can be obtained beforehand that @xmath80 and @xmath159 equal @xmath160 and @xmath161 , respectively .
thus , one has @xmath162 further notice that @xmath163 it can be proved that all of the three items in the square root of ( [ beta1beta2_coro : steady_q ] ) are increasing with respect to @xmath78 and @xmath81 ; thus the second item in the denominator is monotonic increasing with respect to @xmath78 and @xmath81 . till now , the monotonicity of @xmath164 with respect to @xmath78 and @xmath81 has been proved .
last , in the special scenario where @xmath78 exactly equals @xmath6 , it can be obtained that @xmath164 is identical to @xmath165 ; thus @xmath165 is larger than the minimum steady - state msd of the scenario where @xmath78 is less than @xmath6 . in sum ,
corollary [ coro : steady_q ] is proved .      for a sparse system in accordance with ( [ sparse_condition ] ) ,
@xmath114 defined in appendix [ const ] are approximated by @xmath166 substituting @xmath114 in @xmath72 of ( [ msd_opt_eta ] ) , with the approximated expressions above , ( [ msd_simple_sparse_smallmu ] ) is finally derived after calculation .
next we show @xmath164 in ( [ msd_simple_sparse_smallmu ] ) is monotonic increasing with respect to @xmath40 .
since ( [ msd_simple_sparse_smallmu ] ) is equivalent with @xmath167 it can be directly observed from ( [ eta6 ] ) that larger @xmath40 results in larger numerator as well as smaller denominator in ( [ msd_simple_appdendix ] ) , which both contribute to the fact that @xmath164 is monotonic increasing with respect to @xmath40 .
thus , the proof of corollary [ coro : steady_mu ] is arrived .
define function @xmath199 then the roots of @xmath200 are eigenvalues of matrix @xmath201 . from ( [ matrix_a ] )
, it can be shown @xmath202 and @xmath203 where @xmath204 denote the entries of @xmath205 .
thus , we know @xmath206 and @xmath207 , which indicates that one root of quadratic function @xmath200 is within the interval @xmath208 $ ] .
similarly , another root of @xmath200 is in @xmath209 .
thus , it can be concluded that the eigenvalues of @xmath201 are both in @xmath210 and satisfy @xmath211 for large step size scenario of @xmath212 , ( [ lambda3 ] ) and ( [ lambda12_bound_full ] ) yield @xmath213    through comparison between ( [ d_conv_closedform ] ) and ( [ d_conv_closedform_lms ] ) , one can know for large @xmath40 , all the three transient items in msd convergence of @xmath0-lms has faster attenuation rate than lms , leading to acceleration of convergence rate .
when @xmath87 remains a constant while @xmath88 approaches zero , from ( [ w_update ] ) , ( [ l_0gradapprx ] ) , and ( [ l_1gradapprx ] ) , it is obvious that the recursion of @xmath0-lms becomes that of za - lms .
furthermore , one can see that @xmath168 equals @xmath169 . from the definition
, it can be shown @xmath139 is an empty set when @xmath88 approaching zero .
consequently , @xmath170 combine ( [ msd_expression_old ] ) , ( [ quadratic_equ ] ) , and ( [ gsc_sum_l1 ] ) , then after quite a series of calculation , the explicit expression of steady - state msd becomes @xmath171 where @xmath93 is the discriminant of quadratic equation ( [ quadratic_equ ] ) , @xmath172 through a series of calculation , it can be proved that ( [ msd_l1_expression ] ) is equivalent with ( [ shi_sparse_lms_msd ] ) obtained in @xcite .
thus , the steady - state msd in za - lms could be regarded as a particular case of that in @xmath0-lms .
from ( [ r_update ] ) , the update formula is @xmath173    since lms algorithm has fast convergence rate , it is reasonable to suppose most filter tap - weights will get close to the corresponding system coefficient very quickly ; thus , the classification of coefficients @xmath69 could help in the derivation of the convergence situation of @xmath174 .    for @xmath57 , from assumption ( vi ) , ( [ coeff_update ] ) takes the form @xmath175    for @xmath62 the mean convergence is firstly derived and then the mean square convergence is deduced .
take expectation in ( [ h_update ] ) , and combine assumptions ( iii ) , ( v ) , and ( vi ) , one knows @xmath176 since @xmath177 , one can finally get @xmath178 combining ( [ coeff_update ] ) , ( [ mean_conv_sc ] ) and employing assumption ( iii ) , it can be achieved @xmath179    next turn to @xmath64 . from assumption ( iv ) , the following formula can be attained employing the steady state result and first - order taylor expansion @xmath180 where @xmath181 , which is the solution to equation ( [ quadratic_equ ] ) .
finally , with assumption ( iii ) we have @xmath182    considering @xmath183 , and combine ( [ lc_conv]),([mean_conv_sc]),([sc_conv ] ) , and ( [ zc_conv ] ) , one can obtain ( [ d_omega_iter ] ) after a series of derivation .
as for the initial value , since @xmath184 , by definition we have @xmath185 and @xmath186 . thus , lemma [ lemma ] is reached .
the authors wish to thank laming chen and four anonymous reviewers for their helpful comments to improve the quality of this paper .
d. angelosante , j. a. bazerque , and g. b. giannakis , `` online adaptive estimation of sparse signals : where rls meets the @xmath2-norm , '' _ ieee trans . signal process .
58 , no . 7 , pp . 3436 - 3447 , jul .
2010 .
g. mileounis , b. babadi , n. kalouptsidis , and v. tarokh , `` an adaptive greedy algorithm with application to nonlinear communications , '' _ ieee trans .
signal process .
6 , pp . 2998 - 3007 , jun .
2010 .
y. kopsinis , k. slavakis , and s. theodoridis , `` online sparse system identification and signal reconstruction using projections onto weighted @xmath2 balls , '' _ ieee trans .
signal process .
905 - 930 , mar .
2011 .
e. cands , j. romberg , and t. tao , `` robust uncertainty principles : exact signal reconstruction from highly incomplete frequency information , '' _ ieee trans .
inf . theory _ ,
489 - 509 , feb . 2006 .
h. zayyani , m. babaie - zadeh , and c. jutten , `` an iterative bayesian algorithm for sparse component analysis ( sca ) in presence of noise , '' _ ieee trans .
signal process .
4378 - 4390 , oct .
j. jin , y. gu , and s. mei , `` a stochastic gradient approach on compressive sensing signal reconstruction based on adaptive filtering framework , '' _
ieee j. sel .
topics signal process . _ ,
409 - 420 , apr . 2010 .
l. horowitz and k. senne , `` performance advantage of complex lms for controlling narrow - band adaptive arrays , '' _ ieee trans .
acoust . , speech , signal process .
assp-29 , no .
722 - 736 , jun .
1981 .
s. k. zhao , z. h. man , s. y. khoo , and h. r. wu,``stability and convergence analysis of transform - domain lms adaptive filters with second - order autoregressive process , '' _ ieee trans .
signal process .
119 - 130 , jan . 2009 .
a. ahlen , l. lindbom , and m. sternad , `` analysis of stability and performance of adaptation algorithms with time - invariant gains , '' _ ieee trans .
signal process .
103 - 116 , jan . 2004 .
s. c. chan and y. zhou , `` convergence behavior of nlms algorithm for gaussian inputs : solutions using generalized abelian integral functions and step size selection , '' _ journal of signal processing systems _ , vol .
3 , pp . 255 - 265 , 2010 .
s. c. chan and y. zhou , `` on the performance analysis of the least mean m - estimate and normalized least mean m - estimate algorithms with gaussian inputs and additive gaussian and additive gaussian and contaminated gaussian noises , '' _ journal of signal processing systems _ , vol .
81 - 103 , 2010 .
s. koike , `` convergence analysis of a data echo canceler with a stochastic gradient adaptive fir filter using the sign algorithm , '' _ ieee trans .
signal process .
2852 - 2861 , dec . 1995