ai applications , such as robotics and logistics , rely on a variety of disciplines such as logic , probabilistic reasoning , machine learning and mathematical programming .
these applications are often described in a combination of natural and mathematical language , and need to be engineered for the individual application .
formal languages that allow declarative specifications enable re - use and descriptive clarity .
the subfields of ai have contributed many formalisms for the declarative modeling and effective solving of problems . in knowledge representation and constraint programming , for example , languages such asp @xcite and essence @xcite are prominent , which use sat , smt and mip technology . in machine learning and probabilistic reasoning , statistical relational learning systems and probabilistic programming languages such as markov logic @xcite , church @xcite and problog @xcite
are prominent , which use sampling and model counting methodologies .
finally , in mathematical programming and optimization , disciplined programming @xcite and ampl @xcite have been developed .
be that as it may , it is widely accepted that solving real - world ai problems requires an integration of different disciplines .
consider , for example , that a robot may decide its course of action using a sat - based planner , learn about the world using kalman filters , and grasp objects using geometric optimization technology @xcite . but
contemporary declarative frameworks offer little support for such universality : kr and constraint formalisms mostly focus on model generation for discrete problems , probabilistic programming languages do not handle linear and arithmetic constraints , and finally , optimization frameworks work with linear algebra and algebraic constraints to specify the problem and thus are quite different from the high - level descriptions used in the other disciplines and do not support probabilistic or logical reasoning .
what is lacking here is a universal modeling framework that allows us to declaratively specify problems involving _ logic _ and _ constraints _ , _ mathematical programs _ , as well as _
discrete and continuous probability distributions _ in a simple , uniform , modular and transparent manner .
such a framework would greatly simplify the development and understanding of ai systems with integrated capabilities , and would tame the model building process . in this paper , we propose a new declarative framework called _ semiring programming _ ( sp ) that is an attempt to fill this gap .
eugene freuder ( @xcite ) famously quipped :
constraint programming represents one of the closest approaches computer science has yet made to the holy grail of programming : the user states the problem , the computer solves it . "
the underlying idea was summarized in the slogan : @xmath0 the vision of sp builds upon this equation in that : @xmath1 thus a model is expressed in a logical system in tandem with a semiring and a weight function . in its essence , semiring programming rests on three important developments :    * semirings have been successfully shown to unify specifications such as finding a solution versus counting the number of solutions @xcite .
* weighted model counting represents perhaps the simplest approach to reason about weights on possible worlds and elegantly decouples the logical representation from the weight function @xcite .
* probabilistic programming languages like church can appeal to their host languages ( e.g. lisp ) to naturally support composition and abstraction .    as a framework
, sp is set up to allow the modeler to freely choose the logical theory ( syntax and semantics ) and so everything from non - classical logical consequence to real - arithmetic is fair game .
together with a semiring and these weights , a program computes the _
count_. our task will be to show that the usual suspects from ai disciplines , such as sat , csp , bayesian inference , and convex optimization can be : ( a ) expressed as a program , and ( b ) count is a solution to the problem , that is , the count can be a \{_yes _ , _ no _ } answer in sat , a probability in bayesian inference , or a bound in convex optimization .
in other words , the count is shown to provide abstractions for challenging ai tasks .
we will also demonstrate a variety of more complex problems , such as matrix factorization , and one on compositionality .    the fundamental question , then , is this : how are to we provide a semantics for such a programming language ? it is not possible to piggyback on mathematical accounts of optimization problems in an obvious way , because the language is allowed to be richer than what is usually assumed .
that is a central focus of this paper .    at the outset
, we remark that our concern here is on the formal nature and generality of the framework , in terms of its modeling capabilities .
ideas on solvers are left to a discussion , and their realizations to the future .
the framework is developed in a general way , agnostic about the meaning of sentences .
we adopt ( and assume familiarity with ) terminology from predicate logic @xcite .    a _ theory _
@xmath2 is a triple @xmath3 , where @xmath4 is a set of sentences called the _ language _ of the theory , @xmath5 a set called the _ models _ of the theory , and @xmath6 a subset of @xmath7 called the _
satisfiability relation_.    the set @xmath4 is implicitly assumed to be defined over a vocabulary @xmath8 of relation and function symbols , each with an associated arity .
constant symbols are 0-ary function symbols .
every @xmath4-model @xmath9 is a tuple containing a universe @xmath10 , and a relation ( function ) for each relation ( function ) symbol of @xmath8 . for relation ( constant )
symbol @xmath11 , the relation ( universe element ) corresponding to @xmath11 in a model @xmath12 is denoted @xmath13 for @xmath14 and @xmath15 we write @xmath16 to say that @xmath12 _ satisfies _
@xmath17 we let @xmath18 denote @xmath19 .
finally , the set @xmath20 denotes the literals in @xmath21 and we write @xmath22 to refer to the @xmath4-literals that are satisfied at @xmath12 .
this formulation is henceforth used to instantiate a particular logical system , such as fragments / extensions of first - order logic , as well as non - classical logical consequence .
[ ex : prop theory ] suppose @xmath23 is as follows : @xmath4 is obtained using a set of 0-ary predicates @xmath24 and boolean connectives , @xmath5 are @xmath25 assignments to the elements of @xmath24 , and @xmath6 denotes satisfaction in propositional logic via the usual inductive rules .
then , we obtain a _
propositional theory_.    [ ex : minimal models ] define a theory @xmath26 where @xmath4 is the positive horn fragment from a finite vocabulary , @xmath5 the set of propositional interpretations , and for @xmath27 , define @xmath28 iff @xmath29 ( _ i.e. , _
satisfaction in propositional logic ) and for all @xmath30 this theory can be used to reason about the _
minimal models _ of a formula .
semiring programming draws from the conceptual simplicity of weighted model counting ( wmc ) , which we briefly recap here .
wmc is an extension of # sat , where one simply counts the number of models of a propositional formula @xcite . in wmc ,
one accords a weight to a model in terms of the literals true at the model , and computes the sum of the weights of all models .
[ defn wmc ] suppose @xmath31 is a formula from a propositional language @xmath4 with a finite vocabulary , and suppose @xmath5 is the set of @xmath4-models .
suppose @xmath32 is a weight function
. then :- models @xmath33 we simply write @xmath34 to mean @xmath35 . ]
@xmath36 is called the _ weighted model count _ ( wmc ) of @xmath31
.    the formulation elegantly decouples the logical sentence from the weight function . in this sense , it is clearly agnostic about how weights are specified in the modeling language , and thus , has emerged as an assembly language for bayesian networks  @xcite , and probabilistic programs @xcite , among others .
our programming model is based on algebraic structures called _ semirings _ @xcite ; the essentials are as follows :    a ( commutative ) semiring @xmath37 is a structure @xmath38 where @xmath39 is a set called the _ elements _ of the semiring , @xmath40 and @xmath41 are associative and commutative , @xmath42 is the identity for @xmath43 and @xmath44 is the identity for @xmath45    abusing notation , when the multiplication operator is not used , we simply refer to the triple @xmath46 as a semiring .    the structure @xmath47 is a commutative semiring in that for every @xmath48 @xmath49 @xmath50 , @xmath51 and so on .
in essence , the semiring programming scheme is as follows :    * * input * : a theory @xmath52 , a sentence @xmath53 a commutative semiring @xmath37 , and a weight function @xmath54 * * output * : the _ count _ , denoted @xmath55 .
the scope of these programs is broad , and so we will need different kinds of generality . roughly , the distinction boils down to : ( a ) whether the set of models for a formula is _ finite _ or _
infinite _ , and ( b ) whether the weight function can be
_ factorized _ over the literals or not ( in which case the weight function directly labels the models of a theory ) .
the thrust of this section is : ( a ) to show how these distinctions subsume important model generation notions in the literature , and ( b ) providing rigorous definitions for the count operator . in terms of organization , we begin with the finite case , before turning to the infinite ones .
we present an early preview of some of the models considered in figure [ fig : scope ] .    [ cols="<,^,^",options="header " , ]     ( set - logic pl ) + ( set - algebra [ nat , max,*,0,1 ] ) + ( declare - predicate p ( ) ) + ( declare - predicate q ( ) ) + f = ( p or q ) + ( declare - weight ( p 1 ) ) + ( declare - weight ( ( neg p ) 2 ) ) + ( declare - weight ( q 3 ) ) + ( declare - weight ( ( neg q ) 4 ) ) + ( count f )      here , we generalize the wmc  formulation to semiring labels , but also go beyond classical propositional logic .
we say the theory @xmath56 is _ finite _ if for every @xmath57 @xmath58 is finite .
so , a propositional language with a finite vocabulary is a finite theory , regardless of ( say ) standard or minimal models .
similarly , a first - order language with a finite herbrand base is also a finite theory .
[ defn finite amc ] suppose @xmath52 is a finite theory .
suppose @xmath59 is a commutative semiring .
suppose @xmath60 for any @xmath61 we define : @xmath62 if @xmath63 , then the problem is _ factorized _
, where:@xmath64    essentially , as in wmc , we sum over models and take products of the weights on literals but wrt a particular semiring .    [ ex : sat as amc ] we demonstrate sat and # sat . consider a propositional theory @xmath65 where @xmath66 and suppose @xmath67 letting @xmath5 be standard @xmath4-models , clearly @xmath68 and @xmath69 .
suppose for every @xmath70 @xmath71 is function such that @xmath72 for the semiring @xmath73 : @xmath74    consider the semiring @xmath75 instead . then @xmath76    as with definition [ defn wmc ] , the framework is agnostic about the modeling language .
but for presentation purposes , programs are sometimes described using a notation inspired by the smt - lib standard @xcite .
we demonstrate mpe ( most probable explanation ) and wmc .
consider the theory , semiring and weight function @xmath71 from figure [ fig : sat ] , which specifies , for example , a vocabulary of two propositions @xmath11 and @xmath77 @xmath78 and @xmath79 in accordance with that semiring , the weight of a model , say @xmath80 , of the formula @xmath81 is @xmath82 thus , for the semiring @xmath83 , we have : @xmath84 which finds the most probable assignment .
consider the semiring @xmath85 instead . then : @xmath86 which gives us the weighted model count .
extending the discussion in @xcite , we consider a class of mathematical programs where linear constraints and propositional formulas can be combined freely .
see figure [ fig : logic mathemtical programming ] for an example with non - linear objectives .
formally , quantifier - free linear integer arithmetic and propositional logic are specified as the underlying logical systems , and the domains of constants are typed
. the program declares formulas @xmath87 , @xmath88 , and @xmath89 .
the counting task is non - factorized , and our convention for assigning weights to models is by letting the declare - weight directive also take arbitrary formulas as arguments .
of course , true holds in every model , and so , the weight of every model is determined by the evaluation of @xmath90 at the model , that is , for any @xmath91 its weight is @xmath92 .
for example , a model that assigns 1 to x1 and 1 to x2 is accorded the weight @xmath93 computing the count over @xmath94 then yields a model of @xmath95 with the highest value for @xmath90 .    to see this program in action , consider that every model of @xmath95 must satisfy @xmath96 and so must admit @xmath97 and @xmath98 since @xmath99 can only take values @xmath100 , given the constraints , the desired model must assign @xmath101 and @xmath102 to @xmath103 and @xmath104 respectively .
then , its weight is @xmath105    ( set - logic qf_lia;pl ) + ( set - algebra [ nat , max,0 ] ) + ( set - type int=\{1, ... ,10 } ) + ( declare - function x1 ( ) int ) + ( declare - predicate p1 ( ) ) + ... / * declare x2 and p2 analogously * / + f = ( ( p1 or p2 ) = > 3*x1
< = 4 ) + g = ( p2 = > ( 2*x2 < = 5 ) ) + h = ( ( f and g ) and p2 ) + ( declare - weight true x1*x2 ) + ( count h )    encoding finite domain constraint satisfaction problems as propositional satisfiability is well - known . the benefit , then , of appealing to our framework is the ability to easily formulate counting instances :    [ thm : csp ] suppose @xmath106 is a csp over variables @xmath107 , domains @xmath108 and constraints @xmath109 . there is a first - order theory @xmath110 , a @xmath4-sentence @xmath31 and a weight function @xmath71 such that @xmath111 over @xmath112 iff @xmath106 has a solution .
furthermore , @xmath106 has @xmath113 solutions iff @xmath114 over @xmath115    constraints are boolean - valued functions @xcite , and so constraints over @xmath107 can be encoded as @xmath4-sentences .
we omit the details , but do an example .
see figure [ fig : graph coloring ] for a counting instance of graph coloring : @xmath116 determines there is an edge between x and y , node(x ) says that x is a node , and color(x , y ) says that node @xmath117 is assigned the color @xmath118 the actual graph is provided using the formula @xmath119 which declares a fully connected 3-node graph . also , cons is a conjunction of the usual coloring constraints , _
an edge between nodes @xmath117 and @xmath120 means that they can not be assigned the same color .
let @xmath5 be a set of first - order structures for the vocabulary @xmath121 , respecting types from figure [ fig : graph coloring ] .
the interpretation of \{edge , node } is assumed to be the same for all the models in @xmath5 and is as given by @xmath122 basically , then , the models differ in their interpretation of @xmath123 .
one model of @xmath124 ( data and cons ) , for example , is @xmath125 .
the weights of all models is 1 , and so , for @xmath75 we get : @xmath126    to summarize , the following result is easily shown for semiring programs :    suppose @xmath127 .
suppose @xmath128 is a solution to @xmath129 in that @xmath130 for sat and csp , and @xmath131 for the rest .
then for any @xmath132 , there is a @xmath65 , @xmath133 @xmath71 and @xmath134 such that @xmath135    ( set - logic fol ) + ( set - algebra [ nat,+,0 ] ) + ( set - type color=\{r , b , g } ) + ( set - type node=\{1,2,3 } ) + ( declare - predicate node ( node ) ) + ( declare - predicate edge ( node , node ) ) + ( declare - predicate color ( node , color ) ) + n = ( node(1 ) and node(2 ) and node(3 ) ) + e = ( edge(1,2 ) and edge(2,3 ) and edge(3,1 ) ) + data = ( n and e ) + cons = / * coloring constraints ( omitted ) * / + ( declare - weight true 1 ) + ( count ( data and cons ) )      defining measures @xcite on the predicate calculus is central to logical characterizations of probability theory @xcite .
we adapt this notion for semirings to introduce a general form of counting . for technical reasons , we assume that the universe of the semirings is @xmath136 .
( if required , the range of the weight function can always be restricted to any subset of @xmath137 )    [ defn amc measure simple ] let @xmath138 be any semiring and @xmath52 a theory .
let @xmath139 be a @xmath140-algebra over @xmath5 in that @xmath141 is a @xmath140-finite measurable space wrt the measure @xmath142 respecting @xmath143 that is , for all @xmath144 @xmath145 @xmath146 and @xmath147 is closed under complement and countable unions : for all pairwise disjoint countable sets @xmath148 we have @xmath149 moreover , because the spaces are @xmath140-finite , @xmath5 is the countable union of measurable sets with finite measure . then for any @xmath53 @xmath150    [ ex : convex ]
we demonstrate convex optimization .
suppose @xmath65 is a first - order theory only containing constants @xmath151 with domains @xmath152 suppose @xmath153 is a conjunction of formulas of the form @xmath154 , for real numbers @xmath155 , and thus describing a polyhedron .
in other words , for every @xmath156 @xmath157 and so , @xmath12 is a real - valued assignment to @xmath158 in particular , if @xmath159 then @xmath160 is a point inside the polyhedron @xmath31 .
let @xmath139 be a @xmath140-algebra over @xmath5 and so every @xmath161 is a measurable set of points .
suppose @xmath162 is a convex function that we are to minimize .
consider the semiring @xmath163 and a measure @xmath147 such that for any @xmath161 : @xmath164 which finds the infimum of the @xmath165-values across the assignments in @xmath166 then , @xmath167 gives the minimum of the convex function in the feasible region determined by @xmath17    suppose @xmath165 is a concave function that is to be maximized .
we would then use @xmath168 instead , which finds the supremum of the @xmath165-values across assignments in @xmath169 .
more generally , the same construction is easily shown to be applicable for other families of mathematical programming ( _ e.g. , _
non - linear optimization ) as follows :    [ thm math programming ] suppose @xmath170 is a set of real - valued variables .
suppose @xmath171 is the feasible region of an optimization problem of the form @xmath172 for @xmath173 where @xmath174 and @xmath175 .
suppose @xmath176 is a function to be maximized ( minimized ) .
then there is a @xmath177 and @xmath134 such that @xmath178 is the maximum ( minimum ) value for @xmath165 in the feasible region @xmath179    [ ex matrix factorization ] for a non - trivial example , consider the problem of matrix factorization , a fundamental concern in information retrieval and computer vision @xcite . given a matrix @xmath180 , we are to compute matrices @xmath181 and @xmath182 , such that @xmath183 is minimized .
( here , @xmath184 denotes the frobenius norm . ) using real arithmetic , we provide a formulation in figure [ fig : matrix factorization ] .
( free variables are assumed to be implicitly quantified from the outside . )
let @xmath65 be the theory of real arithmetic , where @xmath4 includes the following function symbols : @xmath185 here , @xmath186 is a real - valued function such that @xmath187 and @xmath188 in that @xmath189 is the entry at the @xmath190 row and the @xmath191 column of the matrix @xmath192 ; these entries are specified by @xmath122 letting @xmath193 ( data and f and g ) , the set @xmath194 are those @xmath4-models whose interpretation of @xmath195 is fixed by @xmath122 basically , these models vary in their interpretations of @xmath196 , which determines their interpretations for @xmath197 and @xmath198 . here ,
app computes the product of the matrices left and @xmath199 and err computes the frobenius norm wrt app and input .
let @xmath139 be a @xmath140-algebra over @xmath200 the weight function in figure [ fig : matrix factorization ] determines a measure @xmath147 such that for any @xmath169 : @xmath201 therefore , @xmath202 yields the lowest @xmath198 value ; the model @xmath12 such that @xmath203 is one with the best factorization of matrix @xmath204    ( set - logic lra ) + ( set - algebra [ real , inf,0 ] ) + ( declare - function input ( int , int ) real ) + ... / * declare left , right , app * / + ( declare - function err ( ) real ) + data = / * entries in input matrix ( omitted ) * / + f = app(x ,
y ) = = sum\{e } left(x , e)*right(e , y ) + g = err = = norm(sum\{x , y } input(x , y ) - app(x , y ) ) + ( declare - weight true err ) + ( count ( data and f and g ) )      despite the generality of the above definition , we would like to address the factorized setting for a number of applications , the most prominent being probabilistic inference in hybrid graphical models @xcite .
consider , for example , a joint distribution on the probability space @xmath205 . here
, it is natural to define weights for each random variable separately , prompting a factorized formulation of counting .
more generally , in many robotic applications , such hybrid spaces are common @xcite .
the main idea then is to apply our definition for counting by measures to each variable independently , and construct a measure for the entire space by _ product measures _ @xcite .
a second technicality is that in the finite case , the set of literals true at a model was finite by definition .
this is no longer the case .
for example , suppose @xmath206 is a real - valued variable in a language @xmath4 , and @xmath12 is a @xmath4-model that assigns @xmath207 to @xmath208 then , @xmath209 but also @xmath210 @xmath211 and so on .
thus , for technical reasons , we assume that @xmath4 only consists of constant symbols @xmath212 with fixed ( possibly infinite ) domains @xmath213 ; the measures are defined for these domains .
[ defn measure amc ] let @xmath214 be any semiring , @xmath215 any theory where @xmath216 over fixed domains @xmath217 suppose @xmath218 suppose @xmath219 is a @xmath140-algebra over @xmath220 in that @xmath221 is a @xmath140-finite measurable space wrt the measure @xmath222 respecting @xmath223 ( as in definition [ defn amc measure simple ] ) . define the product measure @xmath224 on the measurable space @xmath225 satisfying :- finite assumption via the hahn - kolmogorov theorem @xcite . ]
@xmath226 for all @xmath227 finally , define @xmath228 ) \end{array}\ ] ] where @xmath229 =    { \left\ { x{_}1\su m \times \ldots \times x{_}k\su m \mid m \in { { \cal m}}(\phi )   \right\}}.   $ ]    intuitively , @xmath230 capture sets of assignments , and the product measure considers the algebraic product of the weights on assignments to terms . as before , for @xmath231 @xmath232 finally , precisely because the measures are defined on the domains of the terms , we obtain these for all of the satisfying interpretations using the construction @xmath233 .
$ ]    [ ex polyhedron ] we demonstrate the problem of finding the volume of a polyhedron , needed in the static analysis of probabilistic programs @xcite .
suppose @xmath65 and @xmath134 is as in example [ ex : convex ] , that is , @xmath31 defines a polyhedron . for every 0-ary function symbol @xmath234 with domain @xmath235
let @xmath236 be the set of all borel subsets of @xmath237 and let @xmath238 be the lebesgue measure .
thus , for any @xmath239 @xmath240 gives the length of this line .
then , for the semiring @xmath241 , the @xmath242 operator sums the lengths of lines for each variable , and @xmath243 computes the products of these lengths .
thus , @xmath244 is the volume of @xmath31 .    to see this in action ,
suppose @xmath245 then @xmath246 , that is , all assignments to @xmath206 and @xmath247 such that @xmath248 is a valid expression in arithmetic .
therefore , @xmath249 = { \left\ { ( n , m ) \mid n\in [ 1,2.5 ] , m \in [ 0,2 ] , n \in { { \mathbb{r } } } , m\in { { \mathbb{r}}}\right\}}.\ ] ] assuming @xmath250 is the lebesgue measure for all borel subsets of the domain of @xmath251 we have @xmath252 , n\in { { \mathbb{r}}}\right\ } } ) = 1.5.\ ] ] analogously , @xmath253 , m\in { { \mathbb{r}}}\right\ } } ) = 2 .
$ ] then , the volume is @xmath254 ) =    1.5 \times 2 = 3 .
$ ]    we demonstrate probabilistic inference in hybrid models @xcite by extending example [ ex polyhedron ] .
consider a probabilistic program : @xmath255 in english : @xmath256 is drawn uniformly from [ 0,1 ] and @xmath257 is the outcome of a coin toss . if @xmath258 and @xmath259 is not 1 , the program terminates successfully .
suppose we are interested in the probability of done , which is expressed as the formula : @xmath260 suppose @xmath65 is the theory of linear real arithmetic , with @xmath261 , @xmath262 and @xmath263 as in example [ ex polyhedron ] , let @xmath264 be the set of all borel subsets of @xmath136 and @xmath265 the lebesgue measure .
let @xmath266 be the set of all subsets of @xmath25 and @xmath267 be the counting measure , _
@xmath268 and @xmath269 .
so @xmath270 and : @xmath271 = { \left\ { ( n , m ) \mid n{>}.6 ,   0\leq n \leq 1 , n\in { { \mathbb{r } } } , m\in { \left\ { 0,1 \right\ } } \right\ } } \cap \\
\quad\quad\quad { \left\ { ( n , m ) \mid 0\leq n\leq 1 , n\in { { \mathbb{r } } } , m\neq 1 , m\in { \left\ { 0,1 \right\ } } \right\}}. \end{array}\ ] ] this means that @xmath272 \right\ } } ) = .4 $ ] and also @xmath273 .
thus , @xmath274 ) = .4\times 1 .
$ ] analogously , @xmath275 ) = 2 .
$ ] therefore , the probability of done is @xmath276    in general , we have a variant of theorem [ thm math programming ] @xcite :    suppose @xmath170 is any set of real - valued variables .
suppose @xmath277 is any countable set .
suppose @xmath278 , where @xmath279 is any region given by conjunctions of expressions of the form @xmath280 and @xmath281 , where @xmath282 , @xmath283 , @xmath284 and @xmath285 then there is a @xmath177 and @xmath134 such that @xmath178 is the volume of @xmath179
a noteworthy feature of many logic - based knowledge representation formalisms is their compositional nature . in semiring programming , using the expressiveness of predicate logic , it is fairly straightforward to combine theories over possibly different signatures ( _ e.g. , _  propositional logic and linear arithmetic ) , as seen , for example , in smt solvers @xcite .
a more intricate flavor of compositionality is when the new specification becomes difficult ( or impossible ) to define using the original components .
this is a common occurrence in large software repositories , and has received a lot of attention in the ai community @xcite .    in this section ,
we do not attempt to duplicate such efforts , but propose a different account of compositionality that is closer in spirit to semiring programming .
it builds on similar ideas for csps @xcite , and is motivated by machine learning problems where learning ( _ i.e. , _  optimization ) and inference ( _ i.e. , _  model counting ) need to be addressed in tandem .
more generally , the contribution here allows us to combine two semiring programs , possibly involving different semirings . for simplicity of presentation
, we consider non - factorized and finite problems .
suppose @xmath286 and @xmath287 are any two semirings .
we define their _ composition _ @xmath288 as :    * @xmath289 ; * @xmath290 ; * for every @xmath291 and @xmath292 , let @xmath293 .
that is , the composition of the semirings is formed from the cartesian product , respecting the summation operator for the individual structures .
[ defn theory compose ] suppose @xmath294 and @xmath295 are theories , where @xmath296 and @xmath297 do not share atoms , @xmath298 and @xmath299 semirings , and @xmath300 and @xmath301 weight functions for @xmath302 and @xmath303 respectively .
given the environments @xmath304 and @xmath305 , we define its _ composition _ as @xmath306 , where @xmath37 is a composition of @xmath298 and @xmath299 and @xmath65 :    * @xmath134 is obtained over boolean connectives from @xmath307 . * @xmath308 * the meaning of @xmath134 is defined inductively : * * @xmath309 for atom @xmath11 iff @xmath310 if @xmath311 and @xmath312 otherwise ; * * @xmath313 iff @xmath314 ; * * @xmath315 iff @xmath316 for @xmath317 . * for any @xmath318 + @xmath319 .
in essence , the cartesian product for the semirings is extended for arbitrary theories and weight functions .
the meaning of formulas rests on the property that @xmath296 and @xmath297 do not share atoms .
it is now easy to see that the counting for problem for @xmath2 works as usual : that is , for any @xmath134 , @xmath320    we demonstrate a ( simple ) instance of combined learning and inference .
imagine a robot navigating a world by performing
_ move _ actions , and believes its actuators need repairs .
but before it alerts the technician , it would like to test this belief . a reasonable test , then
, is to inspect its trajectory so far , and check whether the expected outcome of a move action in the current state matches the behavior of the very first move action .
more precisely , the robot needs to appeal to linear regression to estimate its expected outcome , and query its beliefs based on the regression model .
we proceed as follows .
let @xmath321 be the theory of real arithmetic with @xmath322 .
suppose that by performing a _ move _ action , the robot s position changes from @xmath323 to @xmath324 let @xmath325 be as follows : @xmath326 the idea is that the values of @xmath323 are the explanatory variables in the regression model and @xmath327 are the response variables , that is , the trajectory data is of the form @xmath328 .    consider a weight function @xmath329 that is , the weight of a model @xmath12 is the universe element corresponding to the constant @xmath330 in @xmath12 , analogous to figure [ fig : logic mathemtical programming ] .
( for simplicity , we assume that the coefficients of the regression model are natural numbers . ) for the semiring @xmath331 : @xmath332 in other words , models in @xmath302 interpret @xmath323 as given by the data , and models differ in their interpretation of @xmath333 and thus , @xmath334 for the data in @xmath335 we would have a model @xmath12 where @xmath336 , @xmath337 and @xmath338 , and so @xmath339      let @xmath340 be a propositional theory , where @xmath341 imagine a weight function @xmath301 as follows : @xmath342 and @xmath343 that is , the robot believes that repairs are needed with a higher probability . indeed , for the semiring @xmath344 : @xmath345
let @xmath306 be the composition of the two environments with @xmath65 and @xmath288 .
suppose @xmath346 is as follows : @xmath347 the final conjunct basically checks whether the expected change in position matches what was happening initially , and if not , _ repair _ should be true .    to see definition [ defn theory compose ] in action , observe that for any @xmath348 @xmath349 , the formulas @xmath350 and those involving @xmath323 are interpreted in @xmath351 but _ repair _ in @xmath352 the weight function is as follows .
given @xmath349 and @xmath353 , we have : @xmath354 then the robot can obtain the weight of _ repair _ and @xmath31 using : @xmath355 where , of course , the first argument is the error of the regression model and the second is 0 because @xmath356 is inconsistent .
that can be contrasted to the count below : @xmath357 it is also easy to see that @xmath358    as in wmc @xcite , suppose the robot obtains the probability of a query @xmath359 given @xmath31 using : @xmath360 where the division is carried out by ignoring the regression error .
then the probability of @xmath361 given @xmath31 is 1 .
thus , no repairs are needed .
the upshot of semiring programming is that it encourages us to inspect strategies for a unified inferential mechanism @xcite .
this has to be done carefully , as we would like to build on scalable methodologies in the literature , by restricting logical theories where necessary . in this section
, we discuss whether our programming model can be made to work well in practice .
let us consider two extremes :    * * option 1 : * at one extreme is a solver strategy based on a single computational technique .
probabilistic programming languages , such as church @xcite , have made significant progress in that respect for generative stochastic processes by appealing to markov chain monte carlo sampling techniques .
unfortunately , such sampling techniques do not scale well on large problems and have little support for linear and logical constraints . *
* option 2 : * at the other extreme is a solver strategy that is arbitrarily heterogeneous , where we develop unique solvers for specific environments , that is , @xmath362 pairs .
we believe the most interesting option is in between these two extremes . in other words , to identify the smallest set of computational techniques , and effectively integrate them is both challenging and insightful .
this may mean that such a strategy is less optimal than option 2 for the environment , but we would obtain a simpler and more compact execution model . to that end , let us make the following observations from our inventory of examples :    * * finite versus infinite : * variable assignments are taken from finite sets versus infinite or uncountable sets . * * non - factorized versus factorized : * the former is usually an optimization problem with an objective function that is to be maximized or minimized .
the latter is usually a counting problem , where we would need to identify one or all solutions .
* * compositionality : * locally consistent solutions ( _ i.e. , _  in each environment @xmath363 ) need to be tested iteratively for global consistency .
thus , option 3 would be realized as follows :    * factorized problems need a methodology for effective enumeration , and therefore , advances in model counting  @xcite , such as knowledge compilation , are the most relevant . for finite theories ,
we take our cue from the problog family of languages @xcite , that have effectively applied arithmetic circuits for tasks such as wmc and mpe .
in particular , it is shown in @xcite how arbitrary semiring labels can be propagated in the circuit .
see @xcite for progress on knowledge compilation in csp - like environments .
+ for infinite theories , there is growing interest in effective model counting for linear arithmetic using smt technology @xcite . like in @xcite , however , we would need to extend these counting approaches to arbitrary semirings . * for non - factorized problems , a natural candidate for handling semirings
does not immediately present itself , making this is a worthwhile research direction . appealing to off - the - shelf optimization software @xcite is always an option , but they embody diverse techniques and the absence of a simple high - level solver strategy makes adapting them for our purposes less obvious . in that regard , solvers for _ optimization modulo theories _ ( omt ) @xcite are perhaps the most promising .
omt technology extends smt technology in additionally including a cost function that is be maximized ( minimized ) . in terms of expressiveness ,
csps @xcite and certain classes of mathematical programs can be expressed , even in the presence of logical connectives . in terms of a solver strategy ,
they use binary search in tandem with lower and upper bounds to find the maximum ( minimum ) .
this is not unlike dpll traces in knowledge compilation , which makes that technology the most accessible for propagating arbitrary semiring labels .
* compositional settings are , of course , more intricate . along with omt , and classical iterative methods like _ expectation maximization _ @xcite
, there are a number of recent approaches employing branch - and - bound search strategies to navigate between local and global consistency @xcite . which of these can be made amenable to compositions of sp programs remains to be seen however .
overall , we believe the most promising first step is to limit the vocabulary of the logical language to propositions and constants ( _ i.e. , _
0-ary functions ) , which make appealing to knowledge compilation and omt technology straightforward .
it will also help us better characterize the complexity of the problems that sp attempts to solve . at first glance
, sp is seen to naturally capture # p - complete problems in the factorized setting , both in the finite case @xcite and the infinite one  @xcite . in the non - factorized
setting , many results from omt and mathematical programming are inherited depending on the nature of the objective function and the domains of the program variables @xcite . by restricting the language
as suggested , the applicability of these results can be explored more thoroughly .
semiring programming is related to efforts from different disciplines within ai , and we discuss representative camps .      formal languages for generative stochastic processes , such as church @xcite and blog @xcite , have received a lot of attention in the learning community .
such languages provide mechanisms to compactly specify complex probability distributions , and appeal to sampling for inference .    closely related to such proposals
are probabilistic logic programming languages such as problog @xcite that extends prolog with probabilistic choices and uses wmc for inference @xcite .
in particular , a semiring generalization of problog , called aproblog @xcite , was the starting point for our work and employs a semiring variation of wmc for inference @xcite , but none addresses matrix factorization , convex optimization and compositionality .
the use of semirings in machine learning is not new to aproblog , see _
@xcite , and programming languages such as dyna @xcite .
dyna is based on datalog ; our logical setting is strictly more expressive than datalog and its extensions ( _ e.g. , _  non - horn fragment , constraints over reals ) .
dyna also labels proofs but not interpretations , as would sp ( thus capturing weighted model counting , for example ) .
the constraints literature boasts a variety of modeling languages , such as essence @xcite , among others @xcite .
( see @xcite , for example , for a proposal on combining heterogeneous solvers . ) on the one hand , sp is more expressive from a logical viewpoint as constraints can be described using arbitrary formulas from predicate logic , and we address many problems beyond constraints , such as probabilistic reasoning . on the other hand ,
such constraint languages make it easier for non - experts to specify problems while sp , in its current form , assumes a background in logic .
such languages , then , would be of interest for extending sp s modeling features .
a notable line of csp research is by @xcite and his colleagues @xcite . here
, semirings are used for diverse csp specifications , which has also been realized in a clp framework @xcite . in particular
, our account of compositionality is influenced by @xcite . under some representational assumptions , sp and such accounts
are related , but as noted , sp can formulate problems such as probabilistic inference in hybrid domains that does not have an obvious analogue in these accounts .
closely related to the constraints literature are the techniques embodied in mathematical programming more generally .
there are three major traditions in this literature that are related to sp .
modeling languages such as ampl @xcite are fairly close to constraint modeling languages , and even allow parametrized constraints , which are ground at the time of search .
the field of disciplined programming @xcite supports features such as object - oriented constraints .
finally , relational mathematical programming @xcite attempts to exploit symmetries in parametrized constraints .    from a solver construction perspective , these languages present interesting possibilities . from a framework point of view , however , there is little support for logical reasoning in a general way .
declarative problem solving is a focus of many proposals , including asp @xcite , model expansion @xcite , among others @xcite .
these proposals are ( mostly ) for problems in np , and so do not capture # p - hard problems like model counting and wmc . indeed , the most glaring difference is the absence of weight functions over possible worlds , which is central to the formulation of statistical models .
weighted extensions of these formalisms , _
@xcite , are thus closer in spirit .
the generality of sp also allows us to instantiate many such proposals , including formalisms using linear arithmetic fragments @xcite .
consider omt for example .
omt can be used to express quantifier - free linear arithmetic sentences with a linear cost function , and a first - order structure that minimizes the cost function is sought . from a specification point of view , sp does not limit the logical language , does not require that objective functions be linear , and a variety of model comparisons , including counting , are possible via semirings .
compositionality in sp , moreover , goes quite beyond this technology .
finally , there is a longstanding interest in combining different ( logical ) environments in a single logical framework , as seen , for example , in modular and multi - context systems @xcite . in such frameworks
, it would be possible to get a ilp program and asp program to communicate their solutions , often by sharing atoms .
in our view , section [ sec : combining_theories ] and these frameworks emphasize different aspects of compositionality .
the sp scheme assumes the modeler will formalize a convex optimization problem and a sat problem in the same programming language since they presumably arise in a single application ( _ e.g. , _  a task and motion planner ) ; this allows model reuse and enables transparency .
in contrast , modular systems essentially treat diverse environments as black - boxes , which is perhaps easier to realize . on the one hand
, it would be interesting to see whether modular systems can address problems such as combined inference and learning .
on the other , some applications may require that different environments share atoms , for which our account on compositionality could be extended by borrowing ideas from modular systems .
in a nutshell , sp is a framework to declaratively specify three major concerns in ai applications :      to its strengths , we find that sp is _ universal _ ( in the above sense ) and _ generic _ ( in terms of allowing instantiations to particular logical languages and semirings ) .
thus , we believe sp represents a simple , uniform , modular and transparent approach to the model building process of complex ai applications .
sp comes with a rigorous semantics to give meaning to its programs . in that sense
, we imagine future developments of sp would follow constraint programming languages and probabilistic programming languages in providing more intricate modeling features which , in the end , resort to the proposed semantics in the paper .
perhaps the most significant aspect of sp is that it also allows us to go beyond existing paradigms as the richness of the framework admits novel formulations that combine theories from these different fields , as illustrated by means of a combined regression and probabilistic inference example . in the long term , we hope sp will contribute to the bridge between learning and reasoning .


photoacoustic tomography is an emerging imaging technique that combines the good contrast of optical absorption with the resolution of ultrasound images ( see for instance @xcite ) . in experiments
an object is irradiated by a short - pulsed laser beam .
depending on the absorption properties of the material , some light energy is absorbed and converted into heat .
this leads to a thermoelastic expansion , which causes a pressure rise , resulting in an ultrasonic wave , called photoacoustic signal .
the signal is detected by an array of ultrasound transducers outside the object . using this signal the pressure distribution at the time of the laser excitation
is reconstructed , offering a 3d image proportional to the amount of absorbed energy at each position .
this is the imaging parameter of photoacoustics .
common measurement setups rely on small ultrasound sensors , which are arranged _ uniformly _ along simple geometries , such as planes , spheres , or cylinders covering the specimen of interest ( see for instance @xcite below ) .    for the planar arrangement of point - like
detectors there exist several approaches for reconstruction , including numerical algorithms based on filtered backprojection formulas and time - reversal algorithms ( see for instance @xcite ) .
the suggested algorithm in the present work realizes a fourier inversion formula ( see below ) using the _ non - uniform fast fourier transform _ ( nufft ) .
this method has been designed for evaluation of fourier transforms at non - equispaced points in frequency domain , or non - equispaced data points in spatial , respectively temporal domain .
the prior is called ner - nufft ( non - equispaced range - non uniform fft ) , whereas the latter is called ned - nufft ( non - equispaced data - non uniform fft ) .
both algorithms have been introduced in @xcite .
both nufft methods haven proven to achieve high accuracy and simultaneously reach the computational efficiency of conventional fft computations on regular grids @xcite . this work investigates photoacoustic reconstructions from ultrasound signals recorded at _ non - equispaced _ positions on a planar surface .
to the best of our knowledge , this is a novel research question in photoacoustics , where the use of regular grids is the common choice . for the reconstruction
we propose a novel combination of ned- and ner - nufft , which we call nedner - nufft , based on the following considerations :    1 .
the discretization of the analytic inversion formulas ( see ) contains evaluation at non - equidistant sample points in frequency domain .
2 .   in addition , and this comes from the motivation of this paper , we consider evaluation at non - uniform sampling points .
the first issue can be solved by a ner - nufft implementation : for * 2d * photoacoustic inversion with * uniformly * placed sensors on a measurement * line * such an implementation has been considered in @xcite .
this method was used for biological photoacoustic imaging in @xcite . in both papers the imaging could be realized in 2d because integrating line detectors @xcite have been used for data recording . in this paper , however , the focus is on 3d imaging , because measurements are taken with point sensors .
experimentally , we show the applicability and superiority of the ( ned)ner - nufft reconstruction formula in three spatial dimensions , compared with standard interpolation fft reconstruction .
to be precise , in this paper we conduct _ three dimensional _ imaging implemented using nedner - nufft , with ultrasound detectors aligned _ non - uniformly _ on a measurement plane . to easily assess the effect of a given arrangement ,
also 2d numerical simulations have been conducted , to support the argumentation .
we quantitatively compare the results with other computational imaging methods : as a reference we use the @xmath0-wave toolbox @xcite with a standard fft implementation of the inversion algorithm .
the nedner - nufft yields an improvement of the lateral and axial resolution ( the latter even by a factor two ) .
the outline of this work is as follows :    in section [ sec : reconst ] we outline the basics of the fourier reconstruction approach by presenting the underlying photoacoustic model .
we state the fourier domain reconstruction formula in a continuous setting .
moreover , we figure out two options for its discretization .
we point out the necessity of a fast and accurate algorithm for computing the occurring discrete fourier transforms with non - uniform sampling points .    in section [ sec : nufft ]
we briefly explain the idea behind the nufft .
we state the ner - nufft ( subsection [ subsec : ner_nufft ] ) and ned - nufft ( subsection [ subsec : ned_nufft ] ) formulas in the form we need it to realize the reconstruction on a non - equispaced grid .    in section
[ sec : comp1_ner_nufft ] we discuss the 3d experimental setup .
the ner - nufft is compared with conventional fft reconstruction .
a test chart is used to quantify resolution improvements in comparison to the _ k - wave _ fft reconstruction with linear interpolation . in axial direction
this improvement was about 170% while reducing the reconstruction time by roughly 35% .    in section [ sec : comp2_ned_nufft ]
we then turn to the nedner - nufft in 2d with simulated data , in order to test different sensor arrangements in an easily controllable environment .
an equiangular arrangement turns out to yield an improvement of over 40% compared to the best choice of equispaced sensor arrangement .
furthermore , we use the insights gained from the 2d simulations to develop an equi - steradian sensor arrangement for our 3d measurements .
we apply our nedner - nufft approach on these data and quantitatively compare the outcomes with reconstructions from equispaced data obtained by the ner - nufft approach .
our results show a significant improvement of the already superior ner - nufft .
let @xmath1 be an open domain in @xmath2 , and @xmath3 a @xmath4 dimensional hyperplane not intersecting @xmath5 .
mathematically , photoacoustic imaging consists in solving the operator equation @xmath6=p|_{\gamma\times(0,\infty)}\,,\ ] ] where @xmath7 is a function with compact support in @xmath5 and @xmath8 $ ] is the trace on @xmath9 of the solution of the equation    @xmath10    in other words , the photoacoustic imaging problem consists in identifying the initial source @xmath7 from measurement data @xmath11 .
an explicit inversion formula for @xmath12 in terms of the fourier transforms of @xmath7 and @xmath13 $ ] has been found in @xcite .
let @xmath14 .
assume without loss of generality ( by choice of proper basis ) that @xmath3 is the hyperplane described by @xmath15 .
then the reconstruction reads as follows : @xmath16\left(\bm{k}\right)= \frac{2k_{y}}{\kappa\left(\bm{k}\right)}\mathbf{f}[\mathbf{q}f]\left(\bm k_{\bm x},\kappa\left(\bm{k}\right)\right).\end{aligned}\ ] ] where @xmath17 denotes the @xmath18-dimensional fourier transform : @xmath19\left(\bm{k}\right):=\frac1{(2\pi)^{n/2}}\int\limits_{\mathbb{r}^{d}}{{\mathrm{e}}}^{-{{\mathrm{i}}}\bm{k}\cdot(\bm{x},y)}f(\bm{x})\mathrm{d}\bm{x}\,,\end{aligned}\ ] ] and @xmath20 here , the variables @xmath21 are in @xmath22 , whereas @xmath23 .    for the numerical realization these three steps have to be realized in discrete form
: we denote evaluations of a function @xmath24 at sampling points @xmath25 by @xmath26 for convenience , we will modify this notation in case of evaluations on an equispaced cartesian grid .
we define the @xmath18-dimensional grid @xmath27 and assume our sampling points to be located on @xmath28 , where @xmath29 and write @xmath30 where @xmath31 resp .
@xmath32 are the occurring step sizes .    in frequency domain , we have to sample symmetrically with respect to @xmath33 .
therefore , we also introduce the interval @xmath34 since we will have to deal with evaluations that are partially in - grid , partially not necessarily in - grid , we will also use combinations of and . in this paper , we will make use of discretizations of the source function @xmath7 , the data function @xmath35 and their fourier transforms @xmath36 resp .
@xmath37 .
let in the following @xmath38 denote the @xmath18-dimensional discrete fourier transform with respect to space and time
. by discretizing formula via riemann sums it follows @xmath39 where @xmath40 this is the formula from @xcite .
note that we use the interval notation for the integer multi - indices for notational convenience .
moreover , we also choose the length of the fourier transforms to be equal to @xmath41 in the first @xmath4 dimensions , respectively .
this could be generalised without changes in practice .
now , we assume to sample @xmath35 at @xmath42 , not necessarily uniform , points @xmath43 : then , @xmath44 the term @xmath45 represents the area of the detector surface around @xmath46 and has to fulfil @xmath47 .
note that the original formula can be received from by choosing @xmath48 to contain all points on the grid @xmath49 .
formula can be interpreted as follows : once we have computed the fourier transform of the data and evaluated the fourier transform at non - equidistant points with respect to the third coordinate , we obtain the ( standard , equispaced ) fourier coefficients of @xmath7 .
the image can then be obtained by applying standard fft techniques .
the straightforward evaluation of the sums on the right hand side of would lead to a computational complexity of order @xmath50 .
usually this is improved by the use of fft methods , which have the drawback that they need both the data and evaluation grid to be equispaced in each coordinate .
this means that if we want to compute efficiently , we have to interpolate both in domain- and frequency space . a simple way of doing that is by using polynomial interpolation .
it is used for photoacoustic reconstruction purposes for instance in the _ k - wave _ toolbox for matlab @xcite .
unfortunately , this kind of interpolation seems to be sub - optimal for fourier - interpolation with respect to both accuracy and computational costs @xcite    a regularized inverse k - space interpolation has already been shown to yield better reconstruction results @xcite .
the superiority of applying the nufft , compared to linear interpolation , has been shown theoretically and computationally by @xcite .
this section is devoted to the brief explanation of the theory and the applicability of the non - uniform fourier transform , where we explain both the ner - nufft ( subsection [ subsec : ner_nufft ] ) and the ned - nufft ( subsection [ subsec : ned_nufft ] ) in the form ( and spatial dimensions ) we utilise them afterwards .    the nedner - nufft algorithm used for implementing essentially ( up to scaling factors ) consists of the following steps :    1 .
compute a @xmath4 dimensional ned - nufft in the @xmath51-coordinates due to our detector placement .
2 .   compute a one - dimensional ner - nufft in the @xmath33-coordinate as indicated by the reconstruction formula .
3 .   compute an equispaced @xmath18-dim inverse fft to obtain a @xmath18 dimensional picture of the initial pressure distribution .      with the ner - nufft ( non equispaced range  non - uniform fft ) it is possible to efficiently evaluate the discrete fourier transform at non - equispaced positions in frequency domain .
to this end , we introduce the one dimensional discrete fourier transform , evaluated at non - equispaced grid points @xmath52 : @xmath53 in order to find an efficient algorithm for evaluation of , we use a window function @xmath54 , an oversampling factor @xmath55 and a parameter @xmath56 that satisfy : [ en : psi ]    1 .   @xmath54 is continuous inside some finite interval @xmath57 $ ] and has its support in this interval and 2 .
@xmath54 is positive in the interval @xmath58 $ ] .
then ( see @xcite ) we have the following representation for the fourier modes occurring in ( [ eq : nudft1 ] ) : @xmath59 by assumption , both @xmath54 and @xmath60 are concentrated around @xmath61 .
so we approximate the sum over all @xmath62 by the sum over the @xmath63 integers @xmath0 that are closest to @xmath64 . by choosing @xmath65 and inserting in ,
we obtain @xmath66 here @xmath67 denotes the interpolation length and @xmath68 where @xmath69 is the nearest integer ( i.e. the nearest equispaced grid point ) to @xmath70 .    the choice of @xmath54 is made in accordance with the assumptions above , so we need @xmath54 to have compact support . furthermore , to make the approximation in ( [ eq : ner_nufft ] ) reasonable ,
its fourier transform @xmath71 needs to be concentrated as much as possible in @xmath72 $ ] . in practice
, a common choice for @xmath54 is the kaiser - bessel function , which fulfils the needed conditions , and its fourier transform is analytically computable .
a second major aim of the present work is to handle data measured at non - equispaced acquisition points @xmath73 in an efficient and accurate way .
therefore we introduce the non - equispaced data , @xmath4 dimensional dft @xmath74 the theory for the ned - nufft is largely analogous to the ner - nufft @xcite as described in subsection [ subsec : ner_nufft ] .
the representation is here used for each entry of @xmath75 and inserted ( with now setting @xmath76 ) into formula , which leads to @xmath77 where the entries in @xmath78 are the nearest integers to @xmath79 .
here we have used the abbreviations @xmath80 for the needed evaluations of @xmath54 and @xmath71 .
further remarks on the implementation of the ned- and ner - nufft , as well as a summery about the properties of the kaiser - bessel function and its fourier transform can be found in @xcite .
before we turn to the evaluation of the algorithm we describe the photoacoustic setup .
our device consists of a fp ( fabry prot ) polymer film sensor for interrogation @xcite . a @xmath81 pulsed laser source and a subsequent optical parametric oscillator ( opo )
provide optical pulses .
these pulses have a very narrow bandwidth and can be tuned within the visible and near infrared spectrum .
the optical pulses are then transmitted via an optical fibre .
when the light is emitted it diverges and impinges upon a sample with homogeneous fluence , thus generating a photoacoustic signal .
this signal is then recorded via the fp - sensor head .
the sensor head consists of an approximately @xmath82 thick polymer ( parylene c ) which is sandwiched between two dichroic dielectric coatings .
these dichroic mirrors have a noteworthy transmission characteristic .
light from @xmath83 to @xmath84 can pass the mirrors largely unabated , whereas the reflectivity from @xmath85 to @xmath86 ( sensor interrogation band ) is about 95% @xcite .
the incident photoacoustic wave produces a linear change in the optical thickness of the polymer film . a focused continuous wave laser , operating within the interrogation band ,
can now determine the change of thickness at the interrogation point via fp - interferometry .
we choose two 2d targets for comparison , a star and a usaf ( us air force ) resolution test chart .
both targets are made of glass with a vacuum - deposited durable chromium coating .
the star target has 72 sectors on a pattern diameter of 5 mm , with an unresolved core diameter of @xmath87 .
the targets are positioned in parallel to the sensor surface at a distance of about @xmath88 , and water is used as coupling medium between the target and the sensor .
the interpolation length for the ner - nufft reconstruction is @xmath89 .
the computational times are shown in tab . [
tab : ner - nufft comparison ] showing that the linearly interpolated fft is about 30 % slower than the ner - nufft .
-axis of a star sample , reconstructed using fft and linear interpolation ( top left ) and using the nufft ( top right ) with a square plotted around the center .
the intensity of the reconstructed image along the sidelines of the square is plotted , for both reconstructions .
the purple line indicates the frequency of the star sample.[fig : starphantomreconstruction ] ]    a segment of the reconstructed star target is shown in fig . [
fig : starphantomreconstruction ] .
the intensity is plotted against the sides of an imaginary square ( @xmath90 ) placed around the center of the star phantom .
it is clearly visible , that the fft reconstruction is not able to represent the line pairs , when the density exceeds @xmath91 , corresponding to a resolution of @xmath87 , whereas they are still largely visible for the ner - nufft reconstruction .    [
cols="^,^,^,^",options="header " , ]     for a quantitative comparison of the resolution we use the usaf chart .
it is recorded on an area of @xmath92 sensor points corresponding to @xmath93 with a grid spacing of @xmath94 and a time resolution of @xmath95 . as
yet there is no standardized procedure to measure the resolution of a photoacoustic imaging system .
we proceed similar to @xcite , by fitting a _ line spread function _
( lsf ) and an _ edge spread function _ ( esf ) to the intensities of our reconstructed data . for
the lsf to be meaningful , its source has to approximate a spatial delta function .
this is the case in the @xmath96-axis , since the chrome coating of the usaf target is just about @xmath97 thick .
-mip of an usaf chart reconstruction conducted with fft ( left ) and ner - nufft ( right ) . on the bottom
images the data - sets used for the quantitative resolution are marked .
the black square depicts the 49 @xmath98-coordinates used for the lsf fit along the @xmath96-axis for the axial resolution .
the data for a single point is shown in the bottom right inlay .
the white lines show the intensity fit for the lateral resolution .
the bottom left inlay shows the data for a single white line .
[ fig : usaf - chart ] ]    we fit a cauchy - lorentz distribution to the @xmath96-axis of our reconstructed data for 49 adjacent @xmath98-coordinates . their positions are marked as a black square within the white square , depicted in the bottom images of fig .
[ fig : usaf - chart ] . the reconstruction in the @xmath96-axis for a single point is shown in the bottom right inlay of fig .
[ fig : usaf - chart ] , for 8 points , covering a distance of @xmath99 in the @xmath96-direction , around the maximum intensity .
a fit of the lorentz distribution is shown for both reconstruction methods .
the fwhm ( _ full width , have maximum _ ) of the lorentz distribution , @xmath100 is the parameter @xmath101 .
the output @xmath102 is the intensity in dependence from the @xmath96-axis , and @xmath103 and @xmath104 are fitting parameters .
the average and standard deviation of @xmath101 for both of the 49 datasets are shown in table [ tab : ner - nufft comparison ] .
the line spread function fwhm of the fft - reconstruction turns out to be more than twice as big as the one of the ner - nufft reconstruction .    for the lateral resolution
, there is no target approximating a delta function , so we have to use the esf instead : @xmath105 the @xmath101 here is the fwhm of the associated lsf , and @xmath106 , @xmath107 and @xmath104 are fitting parameters .
the esf requires a step function as source , of which our target provides plenty .
we choose the long side of 12 bars , marked by white lines in the bottom images of fig .
[ fig : usaf - chart ] , for this fit .
the data for a particular line is shown in the bottom left inlay .
we omitted all datasets , where only one point marked the transition from low to high intensity , rendering the edge fit unreliable and resulting in unrealistic improvements of our new method well over 100% .
finally we averaged over 15 edges .
the results are shown in table [ tab : ner - nufft comparison ] . while the deviation between different edges is quite high , the improvement of our ner - nufft reconstruction for the 15 evaluated edges ranged from @xmath108 to @xmath109 .
the current setups allow data acquisition at just one single sensor point for each laser pulse excitation .
since our laser is operating at @xmath81 data recording of a typical sample requires several minutes .
reducing this acquisition time is a crucial step in advancing photoacoustics towards clinical and preclinical application .
therefore , in this work we try to maximize the image quality for a given number of acquisition points .
we are able to do this , because our newly implemented nedner - nufft is ideal for dealing with non - equispaced positioned sensors .
this newly gained flexibility of sensor positioning offers many possibilities to enhance the image quality , compared against a rectangular grid .
for instance a hexagonal grid was found to yield an efficiency of @xmath110 compared with @xmath111 for an exact reconstruction of a wave - number limited function @xcite .
also any non - equispaced grids that may arrive from a specific experimental setup can be efficiently computed via the nedner - nufft approach .    here
, we will use it to tackle the limited view problem .
many papers deal with the limited view problem , when reconstructing images @xcite .
our approach to deal with this problem is different .
it takes into account that in many cases the limiting factor is the number of sensor points and the limited view largely a consequence of this limitation .    in our approach
we therefore use a grid arrangement that is dense close to a center of interest and becomes sparser the further away the sampling points are located .
we realize this by means of an equiangular , or equi - steradian sensor arrangement , where for a given point of interest each unit angle or steradian gets assigned one sensor point .    ]    to understand the limited view problem , it is helpful to define a detection region . according to @xcite , this is the region which is enclosed by the normal lines from the edges of the sensor .
pressure waves always travel in the direction of the normal vector of the boundary of the expanding object .
therefore certain boundaries are invisible to the detector as depicted in fig .
[ fig : fieldofview ] .      for the equiangular sensor arrangement a point of interest is chosen .
each line , connecting a sensor point with the point of interest , encloses a fixed angle to its adjacent line . in that sense
we mimic a circular sensor array on a straight line .
the position of the sensor points can be seen on top of the third image in fig .
[ fig : tree ] .
the obvious expansion of an equiangular projection to 3d is an equi - steradian projection .
this problem is analogous to the problem of placing equispaced points on a 3d sphere and then projecting the points , from the center of the sphere , through the points , onto a 2d plane outside the sphere .
we developed an algorithm for this problem , which is explained in detail in appendix [ app : equi - ster ] .
our input variables are the grid size , the distance of the center of interest from the sensor plane and the desired number of acquisition points , which will be rounded to the next convenient value .
a sensor arrangement with 1625 points on a @xmath112 grid is shown in the top left image in fig .
[ fig : sensor_masks ] .      to determine the weighting term @xmath45 in eq .
[ eq : disc_rec ] for 3d we introduce a function that describes the density of equidistant points per unit area @xmath113 . in our specific case
, @xmath113 describes the density on a sphere around a center of interest .
further we assume that @xmath113 is spherically symmetric and decreases quadratically with the distance from the center of interest @xmath114 : @xmath115 .
we now define @xmath116 for a plane positioned at distance @xmath117 from the center of interest . in this case
@xmath118 attenuates by a factor of @xmath119 , where @xmath120 is the angle of incidence . hence @xmath121 . by applying the spacing of the regular grid .
this yields a weighting term of : @xmath122    analogously we can derive @xmath45 for 2d :    @xmath123    we applied a normalization after the reconstruction to all measurements .    for the application of this method to the fp setup it is noteworthy that there is a frequency dependency on sensitivity which itself depends on the angle of incidence , which has been extensively discussed in @xcite .
the angle of incidence for our specific setup is @xmath124 . at this angle ,
the frequency components around @xmath125 get attenuated by more than @xmath126 .
below @xmath127 the frequency response remains quite stable ( attenuation below @xmath128 for the measurement angles occurring in our setup .
) equispaced sensor arrangement , with a distance of 13 points between each sensor .
the third image shows the nedner - nufft reconstruction with equiangular arranged sensor positions .
the bottom image shows the same sensor arrangement , but all omitted sensor points are linearly interpolated and afterwards a ner - nufft reconstruction was conducted .
[ fig : tree ] ]    a tree phantom , designed by brian hurshman and licensed under cc by 3.0 , is chosen for the 2 dimensional computational experiments on a grid with @xmath129 @xmath130 points .
a forward simulation is conducted via _ k - wave _ @xcite .
the forward simulation of the k - wave toolbox is based on a first order k - space model .
a pml ( perfectly matched layer ) of 64 gridpoints is added , as well as @xmath131 of noise .    in fig .
[ fig : tree ] our computational phantom is shown at the top .
for each reconstruction a subset of 32 out of the 1024 possible sensor positions was chosen . in fig .
[ fig : tree ] their positions are marked at the top of each reconstructed image .
for the equispaced sensor arrangements , we let the distance between two adjacent sensor points sweep from 1 to 32 .
the sensor points where always centered in the @xmath132-axis .    to compare the different reconstruction methods we used the correlation coefficient and the tenenbaum sharpness .
these quality measures are explained in appendix [ app : qualitmeas ] .
we applied the correlation coefficient only within the region of interest marked by the white circle in fig .
[ fig : tree ] .
the tenenbaum sharpness was calculated on the smallest rectangle , containing all pixels within the circle .
the results are shown in fig .
[ fig : tree_cc ] .    .
the straight lines indicate the results for the equiangular projection.[fig : tree_cc ] ]    the tenenbaum sharpness for the equiangular sensor placement was 23001 , which is above all values for the equispaced arrangements .
the correlation coefficient was 0.913 compared to 0.849 , for the best equispaced arrangement .
in other words , the equiangular arrangement is 42.3 % closer to a full correlation , than any equispaced grid .    in fig .
[ fig : tree ] the competing reconstructions are compared . while the crown of the tree is depicted quite well for the equispaced reconstruction ,
the trunk of the tree is barely visible .
this is due to the limited view problem . when the equispaced interval increases , the tree becomes visible , but at the cost of the crown s quality . in the equiangular arrangement a trade off between these two effects is achieved .
additionally the weighting term for the outmost sensors is 17 times the weighting term for the sensor point closest to the middle .
this amplifies the occurrence of artefacts , particularly outside of our region of interest .    the bottom image in fig .
[ fig : tree ] shows the equiangular sensor arrangement , reconstructed in a conventional manner .
the missing sensor points are interpolated to an equispaced grid , and a ner - nufft reconstruction is applied afterwards .
we conducted a linear interpolation from our subset to all 1024 sensor points .
the correlation coefficient for this outcome was 0.7348 while the sharpness measure was 21474 .
this outcome exemplifies the clear superiority of the nufft to conventional fft reconstruction when dealing with non - equispaced grids .
]    for a qualitative assessment of our new sensor arrangement we need a 3d phantom .
we choose a yarn which we record on a rectangular grid with @xmath133 sensor points , with a grid spacing of @xmath134 and time sampling of @xmath135 hence an area of @xmath136 is covered . as coupling medium water
is used , in which the yarn is fully immersed .    to determine the utility of non - equispaced grid sampling , we follow a certain routine .
first we acquire a densely sampled dataset
. then we use a very small subset of the initially collected sensor data , to test different sensor arrangements .
therefore we can always use the complete reconstruction as our model standard together with the quality measurements explained in appendix [ app : qualitmeas ] .
an upsampling factor of 2 was used for all reconstructions , hence the reconstructed image of the mip for the @xmath98 plane consists of @xmath137 pixels .
the complete reconstruction with the ner - nufft took 154 seconds .
maximum intensity projections ( mip ) of this full reconstruction for all axis are shown in fig .
[ fig : golden - standard ] .
the mip in the @xmath98 plane is our model standard , for comparison with the other reconstructions .
sensor points is displayed .
the intervall between two adjacent sensor points is @xmath138 for this configuration .
the bottom images show mips of the ned - ner - nufft reconstruction only using the sensor points shown above . [
fig : sensor_masks ] ]    for the equi - steradian sensor mask we choose our center of interest right in the center of the @xmath98-mip where the little knot can be seen , @xmath139 off the sensor surface
. the resulting sensor mask , including the reconstruction is shown in fig .
[ fig : sensor_masks ] , it consists of @xmath140 sensor points ( or 3.18 % of the initial number of sensor points ) .
the weighting term , accounting for sensor sparsity is 9.7 times higher for the outermost sensor point , than for centermost sensor points in the @xmath98-plane .
the reconstruction for this arrangement took 134 seconds .
we compared this arrangement to rectangular grids , which all had @xmath141 sensor points ( or 3.29 % of the initial number of sensor points ) but varying distances between two adjacent points .
the grid with an interval between sensor points of 5 is shown on the top left in fig .
[ fig : sensor_masks ] .    ) . in the inlay
the correlation coefficient mask is shown for 20087 points which corresponds to all pixels with at least 4% of the maximum value of the model standard and is the value used in fig .
[ fig : correlation - coefficient - maskdiameter ] .
[ fig : correlation_coefficient_mask ] ]    in fig .
[ fig : correlation_coefficient_mask ] the equi - steradian grid is compared with 2 equispaced grids . to get a more precise measure of the correlation between the reconstructed images and the model standard , we calculated the correlation coefficient only within a region of interest .
the region of interest is firstly confined by a centered disc , whose boundary is shown as a dotted circle in the inlay in fig . [
fig : correlation_coefficient_mask ] .
the @xmath132-axis shows the number of pixels of interest used to calculate the correlation coefficient within this disc .
these pixels are increased according to the intensity of the corresponding pixels of the model s standard mip .
[ fig : correlation_coefficient_mask ] demonstrates that the correlation coefficient for the equi - steradian arrangement , always remains better , within the depicted disc , when competing against the two strongest equispaced grids .
corresponds to the straight line in fig .
[ fig : correlation_coefficient_mask ] .
the other straight lines indicate the side length of the square of a particular equispaced sensor point arrangement .
[ fig : correlation - coefficient - maskdiameter ] ]    in fig .
[ fig : correlation - coefficient - maskdiameter ] the 4 equispaced arrangements , with intervals reaching from 2 to 5 are compared to the equi - steradian grid . here
the pixels of interest are set to a threshold of at least 4% of the maximum value , while the diameter is increased .
the dotted straight lines indicate the side length of the square of a particular equispaced sensor point arrangement .
as expected the correlation coefficient for this equispaced sensor arrangement starts to decline around that threshold .
the correlation coefficient for the equi - steradian grid starts to fall behind the equispaced grid of interval 5 towards the end .
this is expected , since the equi - steradian grid is only meant to give better results for a region of interest around the center .
this is very clearly the case . between a diameter of 2 to 8 mm
the correlation coefficient is on average 25.14 % closer to a value of @xmath142 than its strongest equispaced contender of interval 4 , and 30.87 % better than the interval 5 grid . at a diameter of @xmath143
the correlation coefficient for the equi - steradian grid was 0.960 compared to 0.947 for the interval 4 grid and 0.944 for the interval 5 grid .
this is 24.8 % and 28.6% closer to full correlation .    ]
[ fig : tenenbaum - sharpness ] shows the normalized tenenbaum sharpness .
the tenenbaum sharpness , unlike the correlation coefficient , can not be calculated on non - adjacent grid points , therefore it has been calculated on the smallest rectangle , that contains all pixels of interest .
the equi - steradian has the highest sharpness for most values , with the interval 4 grid being very slightly better around a diameter of 3 - 4 mm .
there is a drop of the tenenbaum sharpness towards the end .
we computationally implemented a 3d non - uniform fft photoacoustic image reconstruction , called ner - nufft ( non equispaced range - non uniform fft ) to efficiently deal with the non - equispaced fourier transform evaluations arising in the reconstruction formula .
this method was compared with the _ k - wave _ implemented fft reconstruction , which uses a polynomial interpolation .
the two reconstruction methods where compared using 2d targets .
the lateral resolution showed an improvement of @xmath144 which is in good agreement with the illustrative results for the star target .
the axial resolution showed an improvement of @xmath145 .
the computation time was about @xmath146 less , for the ner - nufft than the linearly interpolated fft reconstruction . in conclusion the ner - nufft reconstruction proved to be unequivocally superior to conventional linear interpolation fft reconstruction methods .
we further implemented the ned - ner - nufft ( non equispaced data - ner - nufft ) , which allowed us to efficiently reconstruct from data recorded at non - equispaced placed sensor points .
this newly gained flexibility was used to tackle the limited view problem , by placing sensors more sparsely further away from the center of interest .
we developed an equiangular sensor placement for 2d and an equi - steradian placement in 3d , which assigns one sensor point to each angle / steradian for a given center of interest . in the 2d computational simulations we showed that this arrangement significantly enhances image quality in comparison to regular grids .
in 3d we conducted experiments , where a yarn phantom was recorded . the maximum intensity projection ( mip ) of the full reconstruction
was compared to mips of reconstructions that only used about 3% of the original data . within our region of interest ,
the correlation of our image was 0.96 , which is 24.8% closer to full correlation than the best equispaced arrangement , reconstructed from slightly more sensor points .
the sensor placement to tackle the limited view problem , combined with the ned - ner - nufft gives significantly better results for an object located at the center of a bigger sensor surface .
this result was confirmed in the 2d simulation as well as for real data in 3d .
this work is supported by the medical university of vienna , the european projects famos ( fp7 ict 317744 ) and fun oct ( fp7 health 201880 ) , macular vision research foundation ( mvrf , usa ) , austrian science fund ( fwf ) , project p26687-n25 ( interdisciplinary coupled physics imaging ) , and the christian doppler society ( christian doppler laboratory `` laser development and their application in medicine '' ) .
[ 1]#10=0=0 0 by1pt#1    10 [ 1]#1 urlstyle [ 1]doi  # 1    p.  beard .
biomedical photoacoustic imaging .
, 1:602631 , 2011 .
p.c . beard .
two - dimensional ultrasound receive array using an angle - tuned fabry - perot polymer film sensor for transducer field characterization and transmission ultrasound imaging .
, 52(6):10021012 , june 2005 .
beard , f.  perennes , and t.n .
transduction mechanisms of the fabry - perot polymer film sensing concept for wideband ultrasound detection . , 46(6):15751582 ,
november 1999 .
p.  burgholzer , c.  hofer , g.  paltauf , m.  haltmeier , and o.  scherzer .
thermoacoustic tomography with integrating area and line detectors .
, 52(9):15771583 , september 2005 .
cox and p.c .
the frequency - dependent directivity of a planar fabry - perot polymer film ultrasound sensor . , 54(2):394404 , 2007 .
w.  dan , c.  tao , x .- j .
liu , and x .- d .
influence of limited - view scanning on depth imaging of photoacoustic tomography .
, 21(1):014301 , 2012 .
k.  fourmont .
non - equispaced fast fourier transforms with applications to tomography . , 9(5):431450 , 2003 .    f.  c.  a. groen , i.  t. young , and g.  ligthart .
a comparison of different focus functions for use in autofocus algorithms .
, 6(2):8191 , 1985 .
m.  haltmeier , o.  scherzer , and g.  zangerl . a reconstruction algorithm for photoacoustic imaging based on the nonuniform fft .
, 28(11):17271735 , november 2009 .
m.  jaeger , s.  schpbach , a.  gertsch , m.  kitz , and m.  frenz .
fourier reconstruction in optoacoustic imaging using truncated regularized inverse k - space interpolation .
, 23:s51s63 , 2007 .
p.  kuchment and l.  kunyansky .
mathematics of thermoacoustic tomography . , 19:191224 , 2008 .
g.  paltauf , r.  nuster , m.  haltmeier , and p.  burgholzer .
photoacoustic tomography with integrating area and line detectors . in l.  v. wang , editor , _ photoacoustic imaging and spectroscopy _ , optical science and engineering , pages 251263 .
crc press , boca raton , fl , 2009 .
d.  p. petersen and d.  middleton .
sampling and reconstruction of wave - number - limited functions in @xmath147-dimensional euclidean spaces .
, 5:279323 , 1962 .
o.  scherzer , editor . .
springer , new york , 2011 .
r.  schulze , g.  zangerl , m.  holotta , d.  meyer , f.  handle , r.  nuster , g.  paltauf , and o.  scherzer . on the use of frequency - domain reconstruction algorithms for photoacoustic imaging .
, 16(8):086002 , 2011 . funded by the austrian science fund ( fwf ) within the fsp s105 - `` photoacoustic imaging '' .    c.  tao and x.  liu
. reconstruction of high quality photoacoustic tomography with a limited - view scanning .
, 18(3):27602766 , feb 2010 .
b.  e. treeby and b.  t. cox .
-wave : matlab toolbox for the simulation and reconstruction of photoacoustic wace fields .
, 15:021314 , 2010 .
b.  e. treeby , t.  k. varslot , e.  z. zhang , j.  g. laufer , and p.  c. beard .
automatic sound speed selection in photoacoustic image reconstruction using an autofocus approach .
, 16(9):0905010905013 , 2011 .
l.  v. wang , editor . . optical science and engineering .
crc press , boca raton , 2009 .
m.  xu and l.  v. wang .
exact frequency - domain reconstruction for thermoacoustic tomography
i : planar geometry . , 21:823828 , 2002 .
m.  xu and l.  v. wang .
time - domain reconstruction for thermoacoustic tomography in a spherical geometry .
, 21(7):814822 , 2002 .
m.  xu and l.  v. wang . analytic explanation of spatial resolution related to bandwidth and detector aperture size in thermoacoustic or photoacoustic reconstruction .
, 67(5):056605105660515 , 2003 .
m.  xu and l.  v. wang .
universal back - projection algorithm for photoacoustic computed tomography . , 71(1 ) , 2005 .
m.  xu and l.  v. wang .
photoacoustic imaging in biomedicine .
, 77(4 ) , 2006 .
y.  xu , d.  feng , and l.  v. wang .
exact frequency - domain reconstrcution for thermoacoustic tomography  i : planar geometry . , 21(7):823828 , 2002 .
y.  xu , l.  v. wang , g.  ambartsoumian , and p.  kuchment .
reconstructions in limited - view thermoacoustic tomography .
, 31(4):724733 , 2004 .
e.  zhang , j.  laufer , and p.  beard .
backward - mode multiwavelength photoacoustic scanner using a planar fabry - perot polymer film ultrasound sensor for high - resolution three - dimensional imaging of biological tissues .
, 47:561577 , 2008 .
in our algorithm , the grid size and the distance of the center of interest from the sensor plane is defined .
the number of sensor points @xmath147 will be rounded to the next convenient value .
our point of interest is placed at @xmath148 , centered at a square @xmath98 grid .
the point of interest is the center of a spherical coordinate system , with the polar angle @xmath149 at the @xmath96-axis towards the @xmath98-grid .
first we determine the steradian @xmath150 of the spherical cap from the point of interest , that projects onto the acquistion point plane via @xmath151 this leads to a unit steradian @xmath152 with @xmath147 being the number of sensors one would like to record the signal with .
the sphere cap is then subdivided into slices @xmath0 which satisfy the condition @xmath153 where @xmath154 encloses exactly one unit steradian @xmath155 and @xmath156 has to be a power of two , in order to guarantee some symmetry . the value of @xmath156 doubles , when @xmath157 , where @xmath158 is the chord length between two points on @xmath0 and @xmath159 is the distance to the closest point on @xmath160 .
these values are chosen in order to guarantee we apply some restrictions to approximate equidistance between acquisition points on the sensor surface .
the azimuthal angles for a slice @xmath0 are calulated according to : @xmath161 with @xmath162 , where @xmath163 stems from the former slice @xmath160 .
the sensor points are now placed on the @xmath98-plane at the position indicated by the spherical angular coordinates : @xmath164
the correlation coefficient @xmath165 is a measure of the linear dependence between two images @xmath166 and @xmath167 .
its range is @xmath168 $ ] and a correlation coefficient close to 1 indicates linear dependence @xcite .
it is defined via the variance , @xmath169 of each image and the covariance , @xmath170 of the two images : @xmath171    we did not choose the widely used @xmath172 distance measure because it is a morphological distance measure , meaning it defines the distance between two images by the distance between their level sets . therefore two identical linearly dependent images can have a correlation coefficient of @xmath142 and still a huge @xmath172 distance .
normalizing the images only mitigates this problem , because single , high intensity artifacts or a varying intensity over the whole image can greatly alter the @xmath172 distance .    while the correlation coefficient is a good measure for the overall similarity between two images it does not include any sharpness measure .
hence blurred edges are punished very little , in comparison to slight variations of homogeneous areas . to address this shortcoming we chose a sharpness measure or focus function as a second quality criterion .
sharpness measures are obtained from some measure of the high frequency content of an image @xcite .
they have also been used to select the best sound speed in photoacoustic image reconstruction @xcite . out of the plethora of published focus functions we select the tenenbaum function , because of its robustness to noise :        like the @xmath175 norm and unlike the correlation coefficient the tenenbaum function is an extensive measure , meaning it increases with image dimensions .
therefore we normalized it to @xmath176 , where @xmath147 is the number of elements in @xmath5 .