spatial discretizations or fourier expansions of the solutions of partial differential equations ( pdes ) which depend on time lead to systems of ordinary differential equations ( odes ) .
the most difficult case arises when the solution of a pde becomes singular in finite time . at such instants
the solution of the pde develops activity down to the zero length scale . a brute force numerical simulation (
no matter how large ) of such a solution is bound to fail because the simulation has a finite resolution and thus will be unable to resolve all the length scales down to the zero scale .
when the solution develops activity at a scale smaller than the smallest scale available to the simulation , the numerically computed solution becomes underresolved .
this leads to a rapid deterioration of the accuracy of the simulation .
the notion of propagation of activity to smaller and smaller scales depends on the physical context of the pde . in some cases ,
e.g. the 3d euler or navier - stokes equations @xcite , this could mean a cascade of energy to smaller and smaller scales .
in other cases , e.g. the nonlinear focusing schrdinger equation @xcite , this could mean a cascade of mass to smaller and smaller scales .
irrespective of the specific physical context , the problem facing the numerical analyst is how to use a _ finite _ simulation and yet prevent the computed solution from suffering a loss of accuracy . in other words , how to construct a numerical method which reproduces correctly the features of the solution of the original equation at the length scales that are available numerically .
this is the motivation behind the construction of reduced models ( see e.g. @xcite ) .    by construction ,
a reduced model must allow for energy ( mass ) to escape from the scales that are accessible to the simulation ( called resolved scales or modes ) to the inaccessible scales ( called unresolved ) .
the main difficulty in constructing an accurate reduced model is the need to estimate the _ correct _ rate at which activity is propagated from the resolved to the unresolved scales .
the mori - zwanzig ( mz ) formalism @xcite proceeds by dividing the available resolution into resolved and unresolved parts .
then , it constructs a reduced model for the resolved scales and uses the unresolved scales to effect the drain of energy ( mass ) out of the resolved scales .
although the mz formalism allows for the construction , in principle , of an exact reduced model it has two drawbacks ( which are also shared by _ any _ other reduction formalism ) .
first , the reduced model can be , in general , prohibitively expensive to calculate .
the reason is that one must obtain an accurate representation of the behavior of the unresolved scales before they can be safely eliminated .
obtaining this representation can be rather costly .
the second drawback is more subtle and has not been adequately appreciated by the scientific computing community .
it is specific to the case of constructing reduced models for singular pdes or in general for systems of ordinary differential equations which are larger than any available numerical resolution .
suppose that you have to construct a reduced model of a full system which is larger than any available numerical simulation .
let us call this system s1 .
exactly because s1 is larger than any available numerical simulation , if we want to construct a reduced model we have to use as a starting point a system , call it s2 , whose size is smaller than the size of s1 .
suppose that you start with s2 and use the mz formalism ( or any other reduction formalism for that matter ) and construct an _ exact _ reduced model s3 for a subset of s2 .
an exact reduced model means that if one evolves s2 and s3 separately , then the behavior of the scales resolved by s3 will be the same as the behavior of the scales in s3 predicted by the simulation of the system s2 .
however , and this is the heart of the problem , since s2 itself will become eventually underresolved , the exact reduced model s3 will also become underresolved .
in other words , the predictions of the exact reduced model s3 can only be trusted for as long as the predictions of the system s2 can be trusted . as a result ,
_ any reduced model that has any chance of being accurate for longer times can not be exact .
_    there are examples of _ inexact _ reduced models , e.g. the @xmath0-model @xcite , coming from the mz formalism , which have been applied to singular pdes and shown numerically to be relatively accurate for long time intervals .
however , the @xmath0-model s accuracy is difficult to assess beforehand and the reason for its relative success has remained a mystery ( see also @xcite for an application to the 3d navier - stokes equations which shows that the @xmath0-model , while not bad , is in need of some modification ) . in order to construct better reduced models we need to incorporate dynamic information from the full system which will help us decide which of the terms appearing in the exact reduced model are the ones that are most important . in this way
, we can construct an inexact but accurate reduced model by keeping the important terms and disregarding the unimportant ones .
the underlying principle of this approach is the same as the principle behind the construction of effective field theories in high energy physics @xcite .    the way we propose to address the problem of constructing better reduced models is to embed the
mz reduced models in a larger class of reduced models which share the same functional form as the mz reduced models but have different coefficients in front of the various terms that appear in the reduced models .
then , one can estimate these coefficients on the fly while the original system of equations is still valid .
the estimation of the coefficients is achieved by requiring that certain integral quantities ( e.g. @xmath1 norms ) involving only resolved scales , should acquire the same values when computed from the original system and the reduced model .
the constraints used to obtain the coefficients are the analog of the  matching conditions " used in effective field theory . also , the approach is the time - dependent analog of the process of renormalization used in high energy and condensed matter physics @xcite .
before the original system ceases to be valid , one reverts to the reduced model with the various coefficients having their estimated values .
we call the proposed approach the renormalized mori - zwanzig algorithm .
in addition to computing the coefficients of the reduced model on the fly , the proposed approach allows us to gauge the importance of the different terms appearing in the reduced model . while at the moment a rigorous proof is lacking , the numerical results ( see section [ universality ] ) suggest that the values of the coefficients depend on the ratio of the smallest scale present in the initial condition to the smallest scale resolved by the reduced model .
the larger is this ratio , the larger are the coefficients for more terms and thus , more terms are needed in the reduced model . on the other hand , for the case when the ratio is small only a few number of terms need to be kept without sacrificing the accuracy of the model .
as we have already said , this situation is analogous to the principle underlying the effective field theory method in high energy physics @xcite .
it is also interesting to investigate to what extent the values of the coefficients are determined by the scaling symmetries of the underlying pde .
this could allow us to identify different universality classes of pdes based on their scaling symmetries ( see also section [ universality ] ) .
finally , we note that the proposed method of computing the coefficients was first presented by the author in @xcite .
the goal in @xcite was to construct a mesh refinement scheme to allow us to reach the singularity instant more efficiently .
then , the computed coefficients were used in a fixed point analysis of the singularity in order to estimate the blow - up exponents . in the current work the goal is different .
we not only want to reach the singularity instant or , more generally , the time that the full system runs out of resolution , but also follow the solution for later times
. also , in the current work , we present a new way of expanding the memory term of the mz formalism .
this allows us , at least for burgers and euler , to calculate recursively and efficiently ( and with minimal storage requirements ) the higher order terms in the memory expansion .
this is a welcome feature since analytical computation of the higher order terms quickly becomes exhausting and , thus , prone to errors .
the paper is organized as follows .
section [ algo ] presents the main construction along with a brief presentation of the mz formalism and how to compute the coefficients of the reduced model on the fly .
section [ numerics ] presents numerical results for the 1d burgers and the 3d incompressible euler equations .
section [ conclusions ] contains a discussion of the results and presents some directions for future work .
in this section we explain the main ideas behind the proposed approach . in section [ full_reduced ]
we set up the notation for the original system and the reduced model in an abstract way which does not make reference to any specific method for obtaining the reduced model . in section [ coefficient_compute ]
we show how to obtain the coefficients for the reduced model . in section [ mzformalism ]
we give a brief presentation of the mz formalism which allows us to obtain the functional form of the terms appearing in the reduced model . in section [ combine ]
we combine the ideas in section [ coefficient_compute ] with the mz formalism from section [ mzformalism ] to derive the proposed algorithm for computing renormalized mz reduced models .
suppose that we want to construct a reduced model for the partial differential equation ( pde ) @xmath2 where @xmath3 is a , in general nonlinear , operator and @xmath4 ( the construction extends readily to the case of systems of partial differential equations ) .
after spatial discretization or expansion of the solution in series , the pde transforms into a system of ordinary differential equations ( odes ) . for simplicity
we restrict ourselves to the case of periodic boundary conditions , so that a fourier expansion of the solution leads to system of odes for the fourier coefficients . to simulate the system for the fourier coefficients we need to truncate at some point the fourier expansion .
let @xmath5 denote the set of fourier modes retained in the series , where we have split the fourier modes in two sets , @xmath6 and @xmath7 we call the modes in @xmath6 resolved and the modes in @xmath8 unresolved .
the reduced model involving only the resolved modes @xmath6 will be called the reduced system and the system involving both the resolved _ and _ unresolved modes @xmath5 will be called the full system .    the main idea behind
the algorithm is that the evolution of moments of the reduced set of modes , for example @xmath1 norms of the modes in @xmath6 , should be the same whether computed from the full or the reduced system .
this requirement will eventually allow us to compute the coefficients appearing in the reduced model ( see section [ coefficient_compute ] ) .
the full system of equations for the modes @xmath5 is given by @xmath9 where @xmath10 is the vector of fourier coefficients of @xmath11 and @xmath12 is the fourier transform of the operator @xmath13 the system should be supplemented with an initial condition @xmath14 the vector of fourier coefficients can be written as @xmath15 where @xmath16 are the resolved modes ( those in @xmath6 ) and @xmath17 the unresolved ones ( those in @xmath8 ) .
similarly , for the right hand sides ( rhs ) we have @xmath18 note that the rhs of the resolved modes involves both resolved and unresolved modes . in anticipation of the construction of a reduced model
we can rewrite the rhs as @xmath19    in general , when one constructs a reduced model , additional terms appear on the rhs of the equations of the reduced model ( see section [ mzformalism ] for more details ) .
the role of these additional terms is to account for the interactions between the resolved and unresolved modes , since the unresolved modes no longer appear explicitly in the reduced model . as is standard in renormalization theory @xcite
, one can augment the rhs of the equations in the full system by including such additional terms .
that is accomplished by multiplying each of these additional terms by a zero coefficient . in this way
, the reduced and full systems rhss have the same functional form .
in particular , for each mode @xmath20 we can rewrite @xmath21 as @xmath22 where @xmath23 and @xmath24 are of the same functional form as the additional terms which appear in the reduced model .
this is easy to do by taking @xmath25 and @xmath26 thus , the equation for the the mode @xmath27 is written as @xmath28 correspondingly , the reduced model for the mode @xmath29 is given by @xmath30 with initial condition @xmath31 we repeat that the functions @xmath32 have the same form as the functions @xmath33 but they depend _ only _ on the reduced set of modes @xmath34 dimensional reduction transforms the vector @xmath35 to @xmath36 this allows one to determine the relation of the full to the reduced system by focusing on the change of the vector @xmath37 to @xmath38 also , the vectors @xmath37 and @xmath39 do not have to be constant in time .
this does not change the analysis that follows .
define @xmath40 quantities @xmath41 involving only modes in @xmath34 for example , these could be @xmath1 norms of the reduced set of modes . to proceed we require that these quantities rates of change are the same
when computed from ( [ full ] ) and ( [ reduced ] ) , i.e. @xmath42 note that similar conditions , albeit time - independent , lie at the heart of the renormalization group theory for equilibrium systems ( @xcite p. 154 ) . also , the conditions are the analog of the  matching conditions " underlying the construction of effective field theories @xcite .      when we only know the functional form of the terms appearing in the reduced model but not their coefficients it is not possible to evolve a reduced system .
we present a way of actually computing the coefficients of the reduced model as needed . if the quantities @xmath41 are e.g. @xmath1 norms of the fourier modes , then we can multiply equations with appropriate quantities and combine with equations to get @xmath43 where @xmath44 are the new rhs functions that appear . note that the rhs of the equations above does not involve primed quantities .
the reason is that here the reduced quantities are computed by using the values of the resolved modes from the full system .
the above system of equations is a linear system of equations for the vector of coefficients @xmath38 the linear system can be written as @xmath45 where @xmath46 this system of equations can provide us with the time evolution of the vector @xmath38    the determination of coefficients for the reduced model through the system is a time - dependent version of the method of moments .
we specify the coefficients of the reduced model so that the reduced model reproduces the rates of change of a finite number of moments of the solution of the original system .
this ensures that each term in the model is properly weighted so that the resulting reduced model reproduces , at the scales accessible to the reduced model , the dynamics ( see ) of the original system .    by construction ,
the entry @xmath47 of the matrix @xmath48 measures the contribution of the @xmath49-th term of the reduced model to the rate of change of @xmath50 in fact , the @xmath49-th column of the matrix @xmath48 is comprised of all the contributions of the @xmath49-th term in the reduced model to the rates of change of the different @xmath50 while the reduced system has no need to transfer activity from the resolved to the unresolved scales , the columns of @xmath48 corresponding to the activity - transferring terms will be zero ( to the numerical precision used ) .
thus , the matrix @xmath48 will be singular .
this can be monitored by estimating the rank of the matrix through the singular value decomposition ( svd ) @xcite .
when the smallest singular value becomes non - zero for the numerical precision used the reduced system starts transferring activity to the unresolved scales .
after that instant we can use the system to estimate the coefficient vector @xmath39 ( see @xcite for more details ) .
the accurate determination of the coefficients guarantees that the rate at which the reduced model transfers activity to the unresolved scales is in agreement with the rate of transfer dictated by the full system .
as we will see in section [ numerics ] , the accurate determination of the coefficients is crucial and moreover it can be hard .
this is because the matrix @xmath48 can turn out to be extremely ill - conditioned .
such ill - conditioning is to be expected because the contributions of the different terms in the reduced model to the rate of change of the quantities @xmath51 can vary dramatically from term to term .
in fact , it is this significant variation of the contributions of the various terms which gauges the importance of the different terms in the reduced model .
eventually , the difference of the importance of the different terms is reflected on the values of the coefficients of the terms .
this will become apparent when we present the numerical results in section [ numerics ] .
we have presented in the previous section an abstract way of writing the reduced system which does not make any reference to a specific method for obtaining the functions @xmath52 appearing on the rhs of . in order to proceed
we need to specify the functions @xmath53 we will do that through the mori - zwanzig formalism @xcite .
suppose we are given the full system @xmath54 where @xmath10 with initial condition @xmath14 the system of ordinary differential equations we are asked to solve can be transformed into a system of linear partial differential equations @xmath55 where @xmath56 the solution of is given by @xmath57 .
using semigroup notation we can rewrite ( [ pde ] ) as @xmath58 suppose that the vector of initial conditions can be divided as @xmath59 where @xmath60 is the vector of the resolved variables and @xmath61 is the vector of the unresolved variables .
let @xmath62 be an orthogonal projection on the space of functions of @xmath60 and @xmath63 we delay the specification of the projection operator until section [ numerics ] where we present applications to specific pdes .
equation can be rewritten as @xmath64 where we have used dyson s formula @xmath65 equation ( [ mz ] ) is the mori - zwanzig identity .
note that this relation is exact and is an alternative way of writing the original pde .
it is the starting point of our approximations .
of course , we have one such equation for each of the resolved variables @xmath66 .
the first term in ( [ mz ] ) is usually called markovian since it depends only on the values of the variables at the current instant , the second is called `` noise '' and the third `` memory '' .    if we write
@xmath67 @xmath68 satisfies the equation @xmath69 if we project ( [ ortho ] ) we get @xmath70 since @xmath71 . also for the initial condition @xmath72 by the same argument .
thus , the solution of ( [ ortho ] ) is at all times orthogonal to the range of @xmath73 we call ( [ ortho ] ) the orthogonal dynamics equation .
since the solutions of the orthogonal dynamics equation remain orthogonal to the range of @xmath62 , we can project the mori - zwanzig equation ( [ mz ] ) and find @xmath74 in order to proceed with the computation of the reduced model we need to compute the markovian term and the memory term . while the markovian term is usually rather straightforward to compute ( see section [ numerics ] ) , the memory term computation is rather involved due to the presence of the orthogonal dynamics evolution operator @xmath75 in fact , it is the presence of this operator which makes , in general , the computation of mz reduced models prohibitively expensive ( see @xcite for a thorough discussion ) .
one can start from and based on assumptions derive simplified reduced models that are easier to calculate @xcite .
however , it has proven difficult to justify the assumptions underlying the derivation of the simplified models .
in addition , it is hard to work with the expression for the memory term given in .
we offer here an alternative presentation of the memory term which allows us to fit the mz formalism in the framework described in section [ full_reduced ] .
note that the memory term @xmath76 can be written , through dyson s formula , as @xmath77 which can be rewritten ( using the linearity of the operator @xmath78 ) as @xmath79 since @xmath80 we find @xmath81 where @xmath82 + \ldots $ ] is given by the bch series and @xmath83=-tltql - tql(-tl)$ ] ( see e.g @xcite ) .
we have @xmath83=[tl , tpl]=[tql , tpl].$ ] in general , @xmath84 \neq 0.$ ] however , depending on the initial conditions , @xmath84 $ ] may be small and thus allow the simplification of the memory term expression .
the magnitude of @xmath84 $ ] will be examined in detail in a future publication .
if we assume that @xmath85 expansion of the operator @xmath86 in taylor series around @xmath87 gives @xmath88 one can obtain different simplified models by truncating the series in after different values of @xmath89 in particular , if we omit all the terms after the first one we obtain the @xmath0-model which has been studied thoroughly @xcite .
the series representation of the memory term in is based on the assumption of analyticity in time of the operator @xmath90 this assumption may be true for small @xmath0 but it does not have to hold for larger @xmath91 in other words , the taylor expansion of the operator @xmath86 has , in general , only a _
finite _ radius of convergence . insisting on using the taylor expansion of the operator @xmath86
as is for later times is dangerous and can lead to the instability of the reduced model ( see also section [ renovsnoreno ] ) .
in fact , when dealing with full systems coming from discretizations of singular pdes , the breakdown of the taylor expansion of the operator @xmath86 is related to the onset of underresolution on the part of the full system .    to proceed we need to put the mz model given by and in the framework of section [ full_reduced ] . to do that we set @xmath92 with this identification we have , in essence ,
embedded the reduced models derived through the mz formalism in a larger class of reduced models which share the same functional form with the mz models but who are allowed to have different coefficients . in the notation of section [ full_reduced ] ,
the original mz models correspond to the coefficient vector @xmath93    while the original mz models may suffer from instabilities , the new models can be made stable by _ assigning to each term in the reduced model the appropriate coefficient_. the magnitude of the coefficient of a term reflects the importance of the term in the reduced model .
the values of the coefficients can now be determined by solving the linear algebraic system .
this ensures that the coefficient of each term in the model is properly redefined ( renormalized ) so that the resulting reduced model reproduces , at the scales accessible to the reduced model , the dynamics ( see ) of the original system .
we are now in a position to state the renormalized mori - zwanzig algorithm which constructs a reduced model with the necessary coefficients computed on the fly .    1
.   choose a number of terms , say @xmath40 , to keep at the taylor expansion of the memory term .
2 .   evolve the full system and compute , at every step , using the svd , the rank of the @xmath94 matrix @xmath95 3 .   when the smallest singular value @xmath96 ( we assume that the singular values are indexed from largest to smallest ) where @xmath97 a prescribed tolerance , solve the system for the coefficients .
4 .   for the remaining simulation time ,
switch from the full system to the reduced model with the estimated values of the coefficients .    in order to apply the algorithm above
, we need to specify the quantities @xmath98 also , we need to compute the expression for the markovian term , as well as the expressions for the terms in the taylor expansion of the memory term . this can be a formidable task .
however , as we will show in the next section , for the case of burgers and euler , these computations can be automated and thus we can compute arbitrarily high order terms in the memory term expansion with minimal effort and storage .
in this section we present results of the rmz algorithm for the 1d burgers and the 3d euler equations .
we use the 1d inviscid burgers equation as an instructive example for the constructions presented in this section .
the equation is given by @xmath99 equation ( [ burgersequation ] ) should be supplemented with an initial condition @xmath100 and boundary conditions .
we solve ( [ burgersequation ] ) in the interval @xmath101 $ ] with periodic boundary conditions .
this allows us to expand the solution in fourier series @xmath102 where @xmath103.$ ] we have written the set of fourier modes as the union of two sets in anticipation of the construction of the reduced model comprising only of the modes in @xmath104,$ ] where @xmath105 the equation of motion for the fourier mode @xmath106 becomes @xmath107 to conform with the mori - zwanzig formalism we set @xmath108 and we have @xmath109 for @xmath110 the system ( [ burgersodemz ] ) is supplemented by the initial condition @xmath111 we focus on initial conditions where the unresolved fourier modes are set to zero .
we also define @xmath112 by @xmath113 note that @xmath114    we also need to define a projection operator @xmath73 for a function @xmath115 of all the variables , the projection operator we will use is defined by @xmath116 i.e. it replaces the value of the unresolved variables @xmath61 in any function @xmath115 by zero .
note that this choice of projection is consistent with the initial conditions we have chosen .
also , we define the markovian term @xmath117 the markovian term has the same functional form as the rhs of the full system but is restricted to a sum over only the resolved modes in @xmath34 the full system conserves the energy @xmath118 contained in all the modes . similarly , the markovian term of the reduced model does _ not _ alter the energy content of the resolved modes .
the necessary energy transfer out of the resolved modes rests on the memory terms .
based on our choice of projection operator and the scaling symmetries of the burgers equation we set @xmath119    with the definition of @xmath62 given above , we find for @xmath120 @xmath121 the expression for @xmath120 contains three terms which involve at least one wavenumber in the unresolved range @xmath7 the terms in the taylor expansion of the memory term are given by @xmath122 for the @xmath49-th term we have @xmath123      the expression for @xmath124 in for the @xmath49th term ( @xmath125 ) can be computed recursively using a simple construction based on a pascal triangle .
note that for our choice of projection operator @xmath126 we have @xmath127 we begin with the ( first - order ) term for @xmath128 i.e. , @xmath129 we find @xmath130 the first order term can be computed by convolving the resolved part of @xmath131 with the unresolved part of @xmath132 in practice , all the convolutions sums can be computed using fast fourier transforms @xcite .
note that the expression @xmath131 is linear in the fourier modes while @xmath133 is quadratic .
thus , the convolution sum in @xmath134 ( including the factor @xmath135)can be denoted as @xmath136 where @xmath137 stands for convolution while @xmath138 and @xmath11 stand for the resolved and unresolved parts .
this notation facilitates the recognition of the pattern for the higher order terms . with this notation
, the first - order term can be written as @xmath139 where we have used bold face to denote the coefficient .
we continue with the second order term @xmath140 we find @xmath141 the convolution sums in this term can be denoted as @xmath142 and @xmath143 the second order term can be written as @xmath144 to see the pattern more clearly we need one more term , @xmath145 we find @xmath146 the terms in the parenthesis can be denoted as @xmath147 @xmath148 and @xmath149 the third order term can be written as @xmath150 by examining the expressions in - we see that the memory terms can be computed as weighted sums of convolution sums where the weights are given by appropriate pascal triangle coefficients ( the bold face numbers ) .
this was to be expected since we started with a convolution sum ( involving products ) of two functions and each new term in the taylor series involves a differentiation .
moreover , the number of convolution sums that need to be added is equal to the order of the memory term in the taylor expansion .
also , for each term , the convolution sums involve expressions whose degree ( in fourier modes ) follows an easily discernible pattern . for the @xmath151th order term in the taylor series
we need the convolution sums @xmath152    finally , the expressions entering the convolution sums can also be computed by a pascal triangle construction .
for example , in order to calculate the third order term , as can be seen from , one needs to compute and store only the quantities @xmath153 for @xmath154 note that the quantities @xmath155 and @xmath156 which are also needed are the same as @xmath157 for the _ resolved _ modes and zero for the _ unresolved _ modes .
so , they do not need to be stored .
they can be quickly constructed when needed .
we see that the storage requirements for the calculation of the memory terms grows only linearly in the order of the taylor expansion .
also , the ability to calculate the needed expressions through ffts speeds up significantly the calculation of the various memory terms .
the recursive estimation of the memory terms allows us to calculate memory terms of very high order efficiently , without having to write down explicitly the analytical expressions which become very complicated after the first few orders in the expansion .
figure [ plot_burgers_collective ] shows the evolution of the energy @xmath158 of the resolved modes computed by reduced models of different orders and the random choice method @xcite .
the initial condition is @xmath159 this leads to the formation of a standing shock at @xmath160 all the reduced models use @xmath161 fourier modes while the full system has @xmath162 modes .
the results of the reduced models are compared to a converged solution of the random choice method with @xmath163 points .
the energy of the random choice method solution was computed using only @xmath161 modes .
however , note that practically all the energy of the random choice method solution is concentrated in the first few fourier modes , so even if we had computed the energy for all @xmath163 fourier methods the results would not have changed .
this is to be expected , since for the initial condition we are using , a standing shock forms at time @xmath164 and , thus , by time @xmath165 the only fourier modes having some energy left in them are the first few .
the quantities @xmath51 used to set up the linear algebraic system needed to compute the coefficients of the reduced system are @xmath1 norms of the solution . in particular , for the first order model we use @xmath166 for the second order model @xmath167 and for the third order model @xmath168 in general , for the reduced model of order @xmath169 we need @xmath170 quantities because we also have to compute the coefficient of the markovian term .
all the calculations are done in double precision .
the tolerance @xmath97 which is used to decide when it is time to switch to the reduced model is set to @xmath171 the systems of ordinary differential equations for the different reduced models were solved using the runge - kutta - fehlberg method with the stepsize control tolerance set to @xmath172 @xcite .
the numerical problem of solving the linear system for the coefficients is hard because the resulting system has very large condition number and very small determinant .
this happens for three reasons .
first , the dominant contribution to the linear system matrix comes from the markovian term ( except for the contribution to the rate of change of the @xmath173 which is zero ) .
this means that the coefficient of the markovian term is practically 1 .
second , the contributions of each memory term to the rates of change of the different @xmath51 vary dramatically .
third , the contributions to each @xmath51 by the different memory terms also varies substantially .
of course , this situation is exacerbated if we use more terms we use in the expansion .
for the case when we retain up to the third order term in the memory expansion , we have to deal with condition numbers of the order @xmath174 and determinant values of order @xmath175 inevitably , even the use of double precision can not provide us with an accurate estimate of the coefficients .
since the linear system matrix is practically singular ( for the numerical precision used ) we have chosen to solve the linear system using the svd algorithm @xcite .
a partial remedy to the problem comes from a slight modification in the way of estimating the coefficients .
since we know that the markovian term coefficient is practically 1 , we can set it to 1 , and subtract the column of contributions of the markovian term from the rhs of the linear system .
this allows us to reduce the dimensionality of the linear system to be solved from @xmath176 to @xmath177 this practice of subtracting almost equal numbers is not advisable in general because it leads to loss of significant digits @xcite .
however , in our case it helps to improve the results by lowering the condition number of the matrix from about @xmath174 to about @xmath178 in figure 1 , the estimation of the coefficients for the third order model using the reduced dimension matrix is denoted as `` rmz - reduced 3rd '' .    as shown in figure [ plot_burgers_collective ] , the rmz models of first and second order give practically the same results as the @xmath0-model .
the third order model gives a slight improvement .
however , when the reduced dimension matrix is used , the energy evolution predicted by the third order model is practically identical to the correct energy evolution of the resolved modes predicted by the random choice method .
if we increase the resolution of the reduced model , the numerical problems for the calculation of higher order coefficients become even more pronounced .
this is to be expected , since a larger resolution means that the renormalized coefficients of the reduced model will be smaller .
thus , computing them with accuracy is more difficult .
we have to note , that the computation of the higher order coefficients is more difficult for our choice of initial condition since it only involves one active fourier mode .
initial conditions with more active fourier modes will transfer activity to the unresolved scales at a higher rate and thus the corresponding renormalized coefficients will be larger .
a detailed study of the behavior of the coefficients for different initial conditions will be presented elsewhere .
for the first order model , we have already presented in @xcite a detailed study about the change of the value of the renormalized coefficient with resolution up to the order of @xmath179 fourier modes . in that work ,
the renormalized coefficient calculation was used to determine , in a fixed point analysis , the blow - up exponent .
consider the 3d incompressible euler equations with periodic boundary conditions in the cube @xmath101 ^ 3 $ ] :    @xmath180    where @xmath181 is the velocity , @xmath182 is the pressure and @xmath183 the system in ( [ euler ] ) is supplemented with the initial condition @xmath100 which is also periodic and incompressible and @xmath184 since we are working with periodic boundary conditions , we expand the solution in fourier series keeping @xmath185 modes in each spatial direction , @xmath102 where @xmath103\times[-\frac{m}{2},\frac{m}{2}-1]\times[-\frac{m}{2},\frac{m}{2}-1].$ ] also @xmath186 and @xmath187    the equation of motion for the fourier mode @xmath106 becomes @xmath188 where @xmath189 is the incompressibility projection matrix and @xmath190 is the @xmath191 identity matrix .
the symbol @xmath192 denotes inner product in @xmath193 the system ( [ eulerode ] ) is supplemented by the initial condition @xmath194 where @xmath195 are the fourier coefficients of the initial condition @xmath196    to conform with the mz formalism we set @xmath197 and we have @xmath198 for @xmath110 the system ( [ eulerodemz ] ) is supplemented by the initial condition @xmath111 note that we focus on initial conditions where the unresolved fourier modes are set to zero .
we also define @xmath199 by @xmath113 note that @xmath114 consider the subset @xmath200\times[-\frac{n}{2},\frac{n}{2}-1]\times[-\frac{n}{2},\frac{n}{2}-1]$ ] for @xmath201 we will construct the reduced models for the fourier modes @xmath106 with @xmath202    we need to define a projection operator @xmath73 for a function @xmath115 of all the variables , the projection operator we will use is defined by @xmath116 i.e. it replaces the value of the unresolved variables @xmath61 in any function @xmath115 by zero .
note , that this choice of projection is consistent with the initial conditions we have chosen .
based on our choice of projection operator and the scaling symmetries of the euler equations we set @xmath119    define @xmath203 the markovian term has the same functional form as the rhs of the full system but is restricted to a sum over only the resolved modes in @xmath34 the full system conserves the energy @xmath118 contained in all the modes .
similarly , the markovian term of the reduced model does _ not _ alter the energy content of the resolved modes .
the necessary energy transfer out of the resolved modes rests on the memory terms .    with the definition of @xmath62 given above
, we find for @xmath120 @xmath204 the expression for @xmath120 contains three terms which involve at least one wavenumber in the unresolved range @xmath7 the terms in the taylor expansion of the memory term are given by @xmath122 for the @xmath49-th term we have @xmath205 the different terms in the memory expansion can be computed recursively as in the case of burgers .
however , there is a slight complication because the presence of the incompressibility operator and of the inner product on the rhs destroys the commutativity which allowed us in burgers to group terms ( the factor 2 which appears outside every parenthesis there ) .
this problem can be addressed by a construction which uses 2 pascal triangles instead of 1 used in the case of burgers . for each order
, one adds up the corresponding terms from the 2 pascal triangles and obtains the desired memory term .
other than that , the recursive algorithm remains the same and we omit the details .
also , note that all the higher order terms are divergence - free by construction .
we have used the same quantities @xmath51 as in the case of burgers , with the obvious generalizations , since for 3d euler we have a 3-dimensional velocity vector instead of the scalar velocity in burgers . also , even though for 3d euler we have a 3-dimensional vector , we have assumed that the reduced model renormalized coefficients are the same for all 3 velocities . this is a simplifying assumption . of course
, one can use different renormalized coefficients for the different velocities at the expense of having to solve a larger linear system for the renormalized coefficients .
a detailed study of that case will be presented in future work .
we have used the taylor - green initial condition ( see e.g. @xcite and references therein ) which is given by @xmath206    figure [ plot_euler_collective ] shows the evolution of the energy @xmath158 of the resolved modes for different resolutions computed by rmz reduced models of different orders and the @xmath0-model .
we have presented results for the rmz reduced models using the reduced linear system matrix approach discussed above to tame the condition number of the matrix . based on these results ,
we make two observations .
first , for @xmath207 resolved modes , the rmz third order model dissipates energy at a slower rate than the @xmath0-model with @xmath207 resolved modes .
this is true not only for the third order model but also for the first and second order models ( we have omitted those results to avoid cluttering the figure )
. this slower rate of energy dissipation compared to the @xmath0-model holds also for the case of @xmath208 resolved modes .
the second observation is that the rate of energy dissipation of the rmz models is consistent with the rate predicted by the @xmath0-model with _ higher _ resolution .
this is to be expected since a higher order model should result in a more accurate prediction of the energy dissipation rate .
the reader may be concerned about the small resolutions used in the numerical experiments .
there are two reasons for that .
first , if one keeps several terms in the memory expansion , then , for a very smooth initial condition like the one we use , the matrix @xmath48 becomes even more ill - conditioned for large resolutions .
however , this is not a severe problem . on the contrary
, it signifies that most of the higher order terms should have small coefficients and thus can be safely removed from the model .
the second reason we have used small resolutions both for burgers and euler is because an accurate reduced model should be able to reproduce the correct energy content for its resolved scales no matter how small the resolution .
for example , for burgers , where we know what the energy content should be after the singularity , we see that the rmz model with a small resolution ( @xmath208 ) indeed reproduces the correct energy content for this resolution .
we have hinted in the introduction at the possibility that the renormalized coefficients may be determined by two factors : i ) the ratio of the smallest active scale in the initial condition to the smallest resolvable scale and ii ) the scaling symmetries of the equation under investigation .
even though the numerical difficulties with the ill - conditioned linear system matrix do not allow us at present to study accurately the higher order renormalized coefficients , we have enough accuracy to study the first order renormalized coefficient both for 1d burgers and 3d euler .
note that the two equations share the same scaling symmetries .
also , we have chosen for burgers the initial condition @xmath209 which has only one active fourier mode ( for @xmath210 ) and for 3d euler , the taylor - green initial condition which also has only active fourier modes for @xmath211    figure [ plot_burgers_euler_coefficient ] shows the comparison of the value of the renormalized first - order coefficient for burgers and 3d euler as a function of the ratio of the smallest scale active in the the initial condition to the smallest scale of the reduced model .
we make two observations .
first , the values of the renormalized coefficient for the two equations are in remarkable agreement .
second , from the slope of the linear fit , we see that the value of the coefficient is practically equal to the ratio of the smallest scale active in the the initial condition to the smallest scale of the reduced model .
needless to say that one example is not enough to infer the generality of the result for arbitrary initial conditions .
a theoretical explanation of this result is lacking at the moment .
note that due to the way we have defined the terms in the expansion of the memory , all the terms have the same dimensions as the markovian term and the left - hand side of the equation for each fourier mode .
so , the corresponding coefficients have to be dimensionless .
thus , we expect the coefficients to depend on ratios of quantities with the same dimensions . here
we have investigated the possibility that this ratio is that of the smallest active scale in the initial condition to the smallest active scale of the reduced model .
we should comment here on the behavior of the rmz algorithm for the 2d euler equations for which the 2d version of the taylor - green initial condition is an exact solution , i.e. a steady state .
exactly because it is a steady state there is no need for a reduced model .
application of the rmz algorithm agrees with this .
there is never any need to transfer energy to the unresolved scales and thus , no need to switch to a reduced model .
the contributions of the different memory terms to the matrix @xmath48 are all well below the double precision threshold .
this allows the freedom to assign to the renormalized coefficient the values shown in figure [ plot_burgers_euler_coefficient ] without incurring any trouble . in other words ,
the behavior of the solution of the 2d euler for the taylor - green initial condition does not contradict the agreement for the renormalized coefficient of the 1d burgers and 3d euler equations shown in figure [ plot_burgers_euler_coefficient ] .      in this section we present results which show that the renormalized version of the mz formalism is advantageous with respect to the original mz formalism . in particular , we show that for the same order in the taylor expansion of the memory term , the renormalized algorithm leads to the stabilization of the reduced model .
figure [ plot_burgers_renovsnoreno ] compares , for the 1d burgers equation , the energy @xmath158 for 16 resolved modes for the renormalized and unrenormalized third order models .
figure [ plot_euler_renovsnoreno ] compares , for the 3d euler equations , the energy @xmath158 for @xmath207 resolved modes for the renormalized and unrenormalized third order models .
the unrenormalized models quickly become unstable and lose all predictive ability .
the unrenormalized expansion leads to divergence of the predicted energy content of the resolved modes .
this is analogous to the divergences that plagued perturbative calculations in quantum field theory ( qft ) before the advent of renormalization @xcite . in qft
, the reason for the divergences was that the perturbation expansion was performed in a quantity ( the bare mass , charge etc . ) which turns out to be ill - defined .
the process of renormalization replaces the perturbation expansion in powers of the ill - defined quantity with a perturbation expansion in powers of the experimentally determined values of this quantity .
this allows the subtraction of the terms that cause divergences and leads to finite results .    in our case ,
the expansion of the memory of the mz formalism is ill - defined because the taylor expansion at @xmath87 breaks down after some time . on the other hand , the renormalized mz formalism takes into account dynamic information from the evolution of the full system ( while this system is still valid ) and prescribes to each term in the memory expansion an appropriate coefficient
. the coefficient measures how important this term is . in this way
, the divergences are averted and the results become finite .
we have presented a new way of computing reduced models for systems of ordinary differential equations .
the approach combines renormalization and effective field theory techniques with the mori - zwanzig formalism .
the constructed reduced models are stable because they transfer activity out of the resolved scales at a rate which is dictated by the full system .
the consistency between the rate of transfer activity of the reduced model and the rate of transfer activity dictated by the full system is the analog of the matching conditions employed in effective field theory .
the matching conditions lead to a redefinition ( renormalization ) of the coefficients of a reduced model originally constructed through the mori - zwanzig formalism .
the results we have obtained for the 1d burgers and 3d euler equations are rather encouraging . the proposed approach of calculating the renormalized coefficients leads to a linear algebraic system for the coefficients .
the matrix of this linear system is ill - conditioned by construction .
the reason is that the reduced model comprises of two parts .
the markovian part which is of the same functional form as the full system but defined only on the resolved scales , and the memory part which has no analog in the full system .
the memory part is there to compensate for whatever activity the markovian term can not reproduce .
thus , the determination of the coefficients appearing in the memory part are , in essence , computed by subtracting the contribution of the markovian term from the full system .
this difference may be small and also the different terms in the memory part can have varying contributions to the difference .
these factors result in the ill - conditioning of the linear system that needs to be solved . in the future
, we plan to address the problem through various techniques designed to deal with ill - conditioned matrices .
the results for 1d burgers and 3d euler equations also point to the possibility that the renormalized coefficients may be completely determined by the structure of the initial condition and the scaling symmetries of the equation .
this needs to be investigated further and a detailed study involving random initial conditions with various spectra will be presented elsewhere .
the proposed approach can also be applied to the navier - stokes equations @xcite .
it is straightforward to incorporate viscosity in the new expansion of the memory term proposed in the current work .
we note that the viscosity starts contributing from the second order memory term .
also , the inclusion of viscosity does not complicate considerably the recursive algorithm for the calculation of the higher order terms .
the expressions needed to compute the viscosity contributions can be estimated through terms already computed in the pascal triangle construction for the inviscid terms .
the approach presented in the current work opens new possibilities for the construction of accurate and stable reduced models for ( large ) systems of ordinary differential equations .
it also highlights the affinity between problems of model reduction in scientific computing and the construction of effective field theories in high energy physics .
we hope that this connection will benefit the problem of constructing reduced models and will be of use in tackling real world problems which are impossible to address through brute force calculations .    in conclusion ,
as steven weinberg once put it @xcite , renormalization is indeed a good thing .
i am grateful to profs . g.i .
barenblatt , a.j . chorin and o.h .
hald for their ongoing guidance and support .