since the mid-1990s we built visual interactive maps of bibliographic and database information at strasbourg astronomical observatory , and some of these , with references , are available at murtagh [ 2006d ] .
the automated annotation of such maps is not easy . at the time of writing zdnet and the bbc ( british broadcasting corporation )
use interactive annotated maps to support information navigation . in zdnet
s case , some prominent terms are graphically presented and can be used to carry out a local search ; and in the bbc case , terms relating to downloadable radio programs are displayed in moving sizes and locations .    in the work described in this article ,
we adopt a different approach : we select the terms of interest in a manual or semi - automated way .
this not only represents expert user judgement but also allows for inclusion of rare or very frequent terms .
in one of our three case studies , we use an automated way to select such terms . for selected terms
, we use their inter - relationships to build a hierarchy and use this as a central device for summarizing information and supporting navigation .
`` ontologies are often equated with taxonomic hierarchies of classes ... but ontologies need not be limited to such a form '' [ gruber 2001 ] .
gruber is cited in gmez - prez et al .  [
2004 ] as characterizing an ontology as `` an explicit specification of a conceptualization '' . in wache
et al .   [ 2001 ] , ontologies are motivated by semantic heterogeneity of distributed data stores .
this is also termed data heterogeneity and is counterposed to structural or schematic heterogeneity .
ontologies are motivated by wache et al .
[ 2001 ] `` for the explication of implicit and hidden knowledge '' , as `` a possible approach to overcome the problem of semantic heterogeneity '' .
so , ontologies may help with integration of diverse , but related , data ; or they may help with clarifying or disambiguating distinctions in the heterogeneous data .
ontologies are likely to be of immediate help in supporting querying . for example , the query model may be based on the ontology ( or ontologies ) used .
there is extensive activity on standards and software , relating more to the above - mentioned schematic rather than semantic heterogeneity , and a useful survey of this area is denny [ 2004 ] .
denny takes an ontology in a broad - ranging view as a knowledge - representation scheme .
a short review of some recent approaches in this area follows .
ahmad and gillam [ 2005 ] develop a semi - automated approach using text with no markup .
multiword expressions are determined , and frequency of occurrence information is used to point to term or phrase importance .
a stop list is used to avoid irrelevant words .
part of speech analysis is not used .
a semantic net is formed to allow development of the ontology elements .
abou assali and zanghi [ 2006 ] use syntactic part of speech tagging to determine the nouns .
these authors retain sufficiently frequent nouns .
they apply the notion of weak subsumption : if  for the most part  a word is in a text that another is in , and not vice versa , then this leads to a hierarchical relationship .
chuang and chien [ 2005 ] assert that multiway trees are appropriate for concept hierarchies , whereas binary trees are built using hierarchical clustering algorithms .
hence they modify the latter to provide more appropriate output .
( a formal approach for mapping a binary hierarchical classification tree onto a multiway hierarchy is described in murtagh [ 2006b ] . )    a hierarchical clustering has often been used to represent an ontology .
note that this is usually not a concept hierarchy .
a concept hierarchy is based on a subsumption relationship between terms , whereas a hierarchical clustering is an embedded set of clusters of the term set . later in this article
( section [ secorientree ] ) , we show a way to derive a concept hierarchy , involving subsumption of terms , from a hierarchic clustering .
a hierarchic clustering is typically a binary , rooted , terminal labeled , ranked tree , and a concept hierarchy is typically a multiway , rooted , terminal and non - terminal labeled , ranked tree . by starting with the former ( binary ) tree representation
, we have an extensive theoretical and formal arsenal at our disposal , to represent the main lines of what we need to do , and to help to avoid ad hoc , user parameter - based , `` engineering '' approaches .
as seen later in this work , we start by laying the foundations of our perspective by basing this on binary trees , and later proceed to the multiway tree .
an alternative approach can be found in ganesan et al .  [
2003 ] , where similarities or distances on trees are redefined and re - axiomatized for the case of multiway trees .
an alternative representation for an ontology is a lattice , and formal concept analysis ( fca ) is a methodology for the analysis of such lattices .
if we have a set of documents or texts , @xmath0 , characterized by an index term set @xmath1 , then as janowitz [ 2005 ] shows , hierarchical clustering and fca are loosely related .
hierarchical clustering is based on pairwise distances or dissimilarities , @xmath2 ( @xmath3 is the set of non - negative reals ) .
fca is based on partially ordered sets ( posets ) such that there is a dissimilarity @xmath4 ( @xmath5 is the power set of the index terms , @xmath1 ) .
other approaches ( rule - based ; machine learning approaches , etc . ; layered , engineering , approaches with maintenance management  see maedche [ 2006 ] ) are also available .
one difficulty with such `` engineering '' approaches is that there is an ad hoc understanding of the problem area , and often there is dependence on somewhat arbitrary threshold and selection criteria that do not generalize well .
our approach formalizes the problem area  the information space  in terms of its local or global topology . where we do have selection criteria , such user interaction is at the application goal level .
visualization is often an important way to elucidate semantic heterogeneity for the user .
visual user interfaces for ontological elucidation are discussed in murtagh et al .
[ 2003 ] , with examples that include interactive , responsive information maps based on the kohonen self - organizing feature map ; and semantic network graphs .
a study is presented in murtagh et al .  [
2003 ] of client - side visualization of concept hierarchies relating to an economics information space .
the use of `` semantic road maps '' to support information retrieval goes back to doyle [ 1961 ] .
motivation , following murtagh et al .
[ 2003 ] , includes the following : ( i ) visualization of the semantic structure and content of the data store allows the user to have some idea before submitting a query as to what type of outcome is possible .
hence visualization is used to summarize the contents of the database or data collection ( i.e. , information space ) .
( ii ) the user s information requirements are often fuzzily and ambiguously defined at the outset of the information search .
hence visualization is used to help the user in his / her information navigation , by signaling the relationships between concepts .
( iii ) ontology visualization therefore helps the user before the user interacts with the information space , and during this interaction .
it is a natural enough progression that the visualization becomes the user interface .
this article is organised as follows .
to begin with , in section [ sect2 ] , we table the issue of whether or not there is inherent hierarchical structure in a text , or a collection of texts . in section [ sect3 ]
we show how we can rigorously determine the extent of inherent hierarchical structure in a text .
this quantifying of inherent hierarchical structure is then used in subsections [ sect41 ] , [ sect42 ] , [ sect52 ] and [ sect7 ] .
a text provides both global and linear semantics , and how we can process these two different perspectives on a given text is discussed in section [ sect4 ] .
a central aspect of our approach is a new distance or metric , which we have recently introduced and exemplified on another data analysis problem .
this new distance is described in subsection [ sect5 ] .    in section [ sect6 ]
we apply what we have described in earlier sections to the selection of salient and characteristic pairs and triplets of terms , and also the selection of pertinent terms .
our motivation is not just the traditional view of phrase counting ( even though we incorporate this view ) but rather the characterization of text content using its internal ( local hierarchical ) structure .
a natural approach to defining a concept hierarchy lies in use of a hierarchical clustering algorithm .
however , the latter forms an embedded sequence of clusters , so that a hierarchy of concepts must  somehow  be derived from it . in section [ secorientree ]
we first of all show that `` converting '' any hierarchical clustering into a hierarchy of concepts is relatively straightforward .
however we do have to face the problem of a unique , and beyond that best , solution .
we show how we can admirably address this need for a unique solution .
our innovative approach is based on the foundations laid in sections [ sect2 ] and [ sect4 ] of this article .
we analyze three different data sets in this work : firstly a set of documents , with some degree of heterogeneity , to illustrate our key goal ; secondly a homogeneous text , partitioned into successive textual segments ; and thirdly a small homogeneous text , partitioned at the sentence level , proxied by lines of text .
we select terms , indeed nouns , in a partially automated way , since this crucial aspect of ontology design may benefit from being user - driven , and may have scalability advantages .
in later sections we address the issue of finding and presenting structure in text .
we link such structure with the textual content .
consequently a key , initial question is to know whether or not there is structure present , and to what extent .
a first problem to be addressed is whether or not the document has any hierarchical structure to begin with . as input , we have possibly a fully tagged document ( based , e.g. , on part - of - speech tagging , schmid [ 1994 ] ) .
however in this work , we start with free text , because it is the most generally available and applicable framework .
additional information provided by part - of - speech tagging can be of use to us , as we will show later .
next we consider the issue of whether or not a document has sufficient inherent hierarchical structure to warrant further investigation . we could approach this problem by fitting a hierarchy , and there are many algorithms for doing so ( such as any hierarchical clustering algorithm ; de soete [ 1986 ] describes a least squares optimal fitting approach ) .
however departure from inherent hierarchical structure is not easily pinpointed .
after all , we have an output induced structure , and we are told , let s say , that the fit is 80% ( defined as @xmath6 where @xmath7 is input dissimilarity , @xmath8 is tree or ultrametric distance read off the output , and the sums are over all pairs ) , which is not very revealing nor useful .
an alternative `` bottom - up '' approach is pursued here , which allows easy assessment of inherent structure , and also pinpointing where this occurs or does not occur .      a formal definition of hierarchical structure is provided by ultrametric topology ( in turn , related closely to p - adic number theory ) .
the triangular inequality holds for a metric space : @xmath9 for any triplet of points @xmath10 .
in addition the properties of symmetry and positive definiteness are respected .
the `` strong triangular inequality '' or ultrametric inequality is : @xmath11 for any triplet @xmath10 .
an ultrametric space implies respect for a range of stringent properties .
for example , the triangle formed by any triplet is necessarily isosceles , with the two large sides equal ; or is equilateral . in an ultrametric space ( i.e. , a space endowed with an ultrametric , or an ultrametric topology ) , one `` lives '' , so to speak , in a tree .
all `` moves '' between one location and another are as if one descended the tree to a common tree node , and then reclimbed to the target point .
topologically , an ultrametric goes a lot further : all points in a circle or sphere are centers , for example ; or the radius of a sphere is identical to its diameter .
the triangle property respected by any triplet of points in an ultrametric space affords a useful way to quantify extent of hierarchical structure .
we will describe our `` extent of hierarchical structure '' , on a scale of 0 ( no respect for ultrametricity ) to 1 ( everywhere , respect for the ultrametric or tree distance ) algorithmically .
we examine triplets of points ( exhaustively if possible , or otherwise through sampling ) , and determine the three angles formed by the associated triangle .
we select the smallest angle formed by the triplet points
. then we check if the other two remaining angles are approximately equal .
if they are equal then our triangle is isosceles with small base , or equilateral ( when all triangles are equal ) .
the approximation to equality is given by 2 degrees ( 0.0349 radians ) .
our motivation for the approximate ( `` fuzzy '' ) equality is that it makes our approach robust and independent of measurement precision .
this approach works very well in practice [ murtagh 2004 ; 2006a ] .
we may note our one assumption for our data when we look at triangles in this way : scalar products define angles so that by assuming our data are in a hilbert space ( a complete normed vector space with a scalar product ) we may proceed with this analysis .
this hilbert space assumption is very straightforward in practice .
when finite ( as is always the case for us , in practice ) , we are using a euclidean space .    often in practice , for arbitrary euclidean data
, there is very little ultrametricity as quantified by the proportion of triangles satisfying the ultrametric requirement .
but recoding the data can be of great help in dramatically increasing the proportion of such ultrametricity - respecting triangles [ murtagh 2004 ; 2005a ] . if we recode our data such that each pairwise distance or dissimilarity is mapped onto one element of the set @xmath12 , then as seen in subsection
[ sect5 ] below the triangular inequality becomes particularly easy to assess for existence of , or non - existence of , a locally ultrametric relationship .
in our use of free text , we have already noted how a mapping into a euclidean space gives us the capability to define distance in a simple and versatile way . in correspondence analysis
[ murtagh 2005b ] , the texts we are using provide the rows , and the set of terms used comprise the column set . in the output , euclidean factor
coordinate space , each text is located as a weighted average of the set of terms ; and each term is located as a weighted average of the set of texts .
( this simultaneous display is sometimes termed a biplot . ) so texts and terms are both mapped into the same , output coordinate space .
this can be of use in understanding a text through its closest terms , or vice versa .
a commonly used methodology for studying a set of texts , or a set of parts of a text ( which is what we will describe below ) , is to characterize each text with numbers of terms appearing in the text , for a set of terms .
the @xmath13 distance is an appropriate weighted euclidean distance for use with such data [ benzcri 1979 ; murtagh 2005b ] . consider texts @xmath14 and @xmath15 crossed by words @xmath16 .
let @xmath17 be the number of occurrences of word @xmath16 in text @xmath14 . then , omitting a constant , the @xmath13 distance between texts @xmath14 and @xmath15 is given by @xmath18 .
the weighting term is @xmath19 .
the weighted euclidean distance is between the _ profile _ of text @xmath14 , viz .
@xmath20 for all @xmath16 , and the analogous _ profile _ of text @xmath15 .
( our discussion is to within a constant because we actually work on _ frequencies _ defined from the numbers of occurrences . )
correspondence analysis allows us to project the space of documents ( we could equally well explore the terms in the _ same _ projected space ) into a euclidean space .
it maps the all - pairs @xmath13 distance into the corresponding euclidean distance .    for a term
, we use the ( full rank ) projections on factors resulting from correspondence analysis .
as noted , this factor space is endowed with the ( unweighted ) euclidean distance .
we will also take into consideration the strongest `` given '' in regard to any classical text : its linearity ( or total ) order .
a text is read from start to finish , and consequently is linearly ordered .
a text endowed with this linear order is analogous to a time series .
if we use the correspondence analysis ( full dimensionality ) factor coordinates for each term , then the textual time series we are dealing with is seen to be a multivariate time series .      just as the way we code our input data plays a crucial role in the resulting analysis , so also the recoding of pairwise distances can influence the analysis greatly . in murtagh
[ 2005a ] we introduced a new distance , which we will term the `` change versus no change '' , cvnc , metric , and showed its benefits on a wide range of ( financial , biomedical , meteorological , telecoms , chaotic , and random ) time series .
motivation for using this new metric is that it greatly increases the ultrametricity of the data .
the cvnc metric is defined in the following way .
take the euclidean distance squared , @xmath21 for all @xmath22 , where we have terms @xmath23 in the factor space with coordinates @xmath24 .
it will be noted below in this section how this assumption of euclidean distance squared has worked well but is not in itself important : in principle any dissimilarity can be used .
we enforce sparseness on our given squared distance values , @xmath25 .
we do this by approximating each value @xmath26 , in the range @xmath27 , by an integer in @xmath28 . the value of @xmath29 must be specified . in our work
we set @xmath30 .
the recoding of distance squared is with reference to the mean distance squared : values less than or equal to this will be mapped to 1 ; and values greater than this threshold will be mapped to 2 . thus far , the recoded value , @xmath31 is not necessarily a distance . with the extra requirement that @xmath32 whenever @xmath33 it can be shown that @xmath31 is a metric [ murtagh 2005a ] :    * theorem : * _
the recoded pairwise measure , @xmath34 , defined as described above from any dissimilarity , is a distance , satisfying the properties of : symmetry , positive definiteness , and triangular inequality . _    to summarize , in our coding ,
a small pairwise dissimilarity is mapped onto a value of 1 ; and a large pairwise dissimilarity is mapped onto a value of 2 .
identical values are unchanged : they are mapped onto 0 .    this coding can be considered as encoding pairwise relationships as `` change '' , i.e.  2 , versus `` no change '' , i.e.  1 , relationships .
then , based on these new distances , we use the ultrametric triangle properties to assess conformity to ultrametricity .
the proportion of ultrametric triangles allows us to fingerprint our data .    for any given triplet ( of terms , with pairwise cvnc distances ) , if the triplet is to be compatible with the ultrametric inequality , each set of three recoded distances is necessarily of one of the following patterns :    trivial : : :    at least one ( recoded ) distance is 0 , in which case we do not consider    it .
equilateral : : :    recoded distances in the triplet are 1,1,1 or 2,2,2 , defining an    equilateral triangle .
ultrametric  isosceles : : :    recoded distances in the triplet are 1,2,2 in any order , defining an    isosceles triangle with small base .
non - ultrametric : : :    recoded distances in the triplet are 1,1,2 in any order .
the non - ultrametric case here is seen to be an isosceles triangle with large base .
we could `` intervene '' and change one of the values to make it ultrametric .
if we change the 2-value to a 1-value , this will produce an equilateral triangle , which is ultrametric . in this case , we are approximating our three values optimally from below , and the resulting ultrametric is termed the subdominant , or maximally inferior , ultrametric . the associated stepwise algorithm for constructing a hierarchy is known as the single link hierarchical clustering algorithm . on the other hand , we could change one of the 1-values to a 2 .
this is not unique , since we could change either of the 1-values .
the resulting hierarchy is termed the minimally superior ultrametric .
the associated stepwise algorithm for constructing a hierarchy is known as the complete link hierarchical clustering algorithm .
all of this is very clear from the case considered here .
the recoding into the cvnc metric is a particular example of symbolic coding .
see murtagh [ 2006c ] .    in the next section
, we will show the usefulness of this cvnc metric for quantifying inherent hierarchical structure .
in this section we first describe the data set used . next ,
based on the foundation of the previous section , we quantify inherent hierarchical structure in our data .
this justifies going further , to harness and exploit this structure .
we use 14 texts taken from wikipedia ( mid-2006 ) , and coverted to straight text from html .
table [ tabprop ] shows the numbers of words in each .
.properties of texts used .
[ cols= " < , < , > , > , > " , ]      to the extent that our data satisfies , globally and throughout , the ultrametric inequality , we can adopt any of the widely used hierarchical clustering algorithms ( single , complete , average linkage ; minimum variance , median , centroid ) to induce an identical , unique hierarchy .
but when we find our data to be , say , 86% ultrametric , as is not untypically the case in practice , then we must consider carefully what our aim is . if we wished to look at each and every isosceles triangle
, then in the case of the artificial intelligence text this means , out of a total of 2,027,795 triplets ( i.e. , @xmath35 ) we must consider 1,007,597 .    what we will do instead is return to taking our text as a time series .
we have 231 unique nouns in the artificial intelligence text . in the text ,
these nouns are used , in total , on 405 occasions .
so our text is a time series of 405 values . for successive nouns in this textual time
series , the cvnc metric has an evident meaning : _ we are noting semantic change versus lack of change as we read through the text_.    we examine successive triplets in the textual time series . for the artificial intelligence text
, we find 45% of the triplets to be equilateral ; 37% of the triplets are isosceles ; and 18% of the triplets are non - ultrametric .
the isosceles triplets point to a dominance or subsumption relationship that will be of use for us in a concept hierarchy .
say we have a triplet @xmath36 .
say , further , that the cvnc distance between @xmath37 and @xmath38 is 1 , so therefore there is no change in progressing from use of term @xmath37 to use of term @xmath38 .
however both @xmath37 and @xmath38 are at cvnc distance 2 to term @xmath39 , and this betokens a semantic change .
so the relationship is simply represented as @xmath40 .
the term @xmath39 dominates or subsumes @xmath37 and @xmath38 .
the following results hold .
firstly , say that a successive triplet of values , in any order , is found as @xmath36 , and later in the text , again , this triplet is found in any order .
then the relationship between the three recoded distances in both cases will be identical . _ for a given triplet , in any order within the triplet , the relationship is unique . _
secondly , consider any other term , @xmath41 , such that some or all of the terms @xmath36 are found to have a relationship with @xmath41 .
as an example , we meet with @xmath42 at one point in the text , and later we meet with @xmath43 . then there is no influence by @xmath41 on the relationship ensuing from the @xmath43 triplet , vis -  - vis the relationship ensuing from the earlier @xmath42 triplet .
_ we have locality of the relationship in any given triplet , from successive terms .
the relationship is strictly local to the given triplet . _    among the isosceles triangles in the artificial intelligence text , we find the following relationships .    ....
( computer science ) branch   ( home computer ) world   ( analysis systems ) formalism   ( analysis systems ) reasoning   ( expert system ) conclusion   ( expert system ) amounts   ( example networks ) reasoning   ( networks learning ) reasoning   ( pattern recognition ) capabilities   ( control systems ) computation   ( consciousness systems ) logic   ( medicine computer ) commentators   ( computer technology ) commentators   ( application feature ) os   ( application feature ) languages   ( libraries systems ) specialist   ( libraries systems ) programmers   ( software engineering ) development   ( software engineering ) practices   ( programs example ) logic   ( example type ) logic   ( projects publications ) life   ( publications bayesian ) life   ( bayesian networks ) life   ( cybernetics systems ) agents   ( systems control ) agents   ( wiki web ) website   ( wiki web ) category   ( algorithm implementations ) projects   ( algorithm implementations ) demonstrations   ( implementations research ) demonstrations   ( research group ) demonstrations   ....    however there are other isosceles triplets that are less self - evident . for this reason
therefore we take all texts . for the 14 texts , we have 6439 nouns , and 1470 unique nouns . with our cvnc metric on all pairs of nouns
, the complete link hierarchical clustering method gives 21 clusters in all .
while one application of the foregoing is to deriving common pairs and triplets of terms , in practice it would be better to combine all relationships into a `` bigger picture '' .
we will address this below in section [ secorientree ] .
presenting a result with around 1500 terms does not lend itself to convenient display .
we ask therefore what the most useful  perhaps the most discriminating terms  are . in correspondence analysis both texts and their characterizing terms are projected into the same factor space .
see figure [ caout ] .
so , from the factor coordinates , we can easily find the closest term(s ) to a given text .
we do this for each of the 14 texts , and find the closest terms , respectively , as follows :    ....       bayesian          automaton     brain           captcha       psychologists     image         maps            logic       topologies        databases     agents          representation          game              games ....    a hierarchical clustering of these is shown in figure [ den15 ] .
the ward minimum variance method is used , as being appropriate for structuring data well ( see murtagh [ 1984b ] ) and also having an agglomerative criterion that is appropriate for the prior euclidean embedding ( viz . , inertia - based in both cases ) .
the data clustered are exactly those illustrated in the best planar projection of figure [ caout ] : these are 14 texts in a 4048-dimensional term space . due to centering in the dual spaces ,
the inherent dimensionality of both text and term spaces are : min@xmath44 = 13 .
based on the dual spaces , we carry out the eigen - reduction in the space of smaller original dimensionality ( viz . , the space of the terms , which are in a 14-dimensional space ) , and then subsequently project into the 4048-dimensional space .
proceeding further , the 5 closest terms to any given text , based on the full inherent dimensionality of this data ( viz . , smaller of dimensionality of texts , and dimensionality of terms ) , are as follows .
.... text and set of 5 closest characterizing terms :    artificial intelligence   bayesian intelligence consciousness brains chatterbots    artificial life automaton automata biology chemical allelomimesis    artificial neural networks brain prediction forecasting aircraft epitomes    captcha captcha captchas robot intelligence chemistry    computational linguistics psychologists logics morphology pragmatics logicians    computer vision image images diagnosis dimensionality dimensions    evolutionary computation maps intelligence robot biology chemistry    fuzzy logic logic mapping animals brakes armies    genetic algorithms topologies communications music finance representations    machine translation databases chemistry database memory robot    multi - agent system agents agent robotics cybernetics robot    semantic network representation database map namespaces robot    turing test game chatterbot consciousness memory intelligence     virtual world games gameplay topography communication representations ....
we have noted in the introduction how a hierarchical clustering may be the starting point for creating a concept hierarchy , but the two representations differ . in this section
we show how we can move from an embedded set of clusters , to an oriented tree .
orientation in the latter case aims at expressing subsumption .
consider the dendrogram shown in figure [ den1 ] , which represents an embedded set of clusters relating to the 8 terms .
we will consider first such a strictly 2-way hierarchy , where we assume that no two agglomerations take place at precisely the same level . in the later case study , in subsection [ sect7 ]
, we will consider the practical case where agglomerations take place at the same level .    rather than the 14 texts used in section [ sect6 ] , to clarify the presentation in this section we will take just one text .
we took aristotle s _ categories _ , which consisted of 14,483 individual words .
we broke the text into 24 files , in order to base the textual analysis on the sequential properties of the argument developed . in these 24 files
there were 1269 unique words .
we selected 66 nouns of particular interest . a sample ( with frequencies of occurrence ) : man ( 104 ) , contrary ( 72 ) , same ( 71 ) , subject ( 60 ) , substance ( 58 ) , ... no stemming or other preprocessing was applied . for the hierarchical clustering
, we further restricted the set of nouns to just 8 .
( these will be seen in the figures to be discussed below . )
the data array was doubled [ murtagh 2005b ] to produce an @xmath45 array , which with removing 0-valued text segments ( since , in one text segment , none of our selected 8 nouns appeared ) gave an @xmath46 array , thereby enforcing equal weighting of ( equal masses for ) the nouns used .
the spaces of the 8 nouns , and of the 23 text segments ( together with the complements of the 23 text segments , on account of the data doubling ) are characterized at the start of the correspondence analysis in terms of their frequencies of occurrence , on which the @xmath13 metric is used .
the correspondence analysis then `` euclideanizes '' both nouns and text segments .
we used a 7-dimensional ( corresponding to the number of non - zero eigenvalues found ) euclidean embedding , furnished by the projections onto the factors .
a hierarchical clustering of the 8 nouns , characterized by their 7-dimensional ( euclidean ) factor projections , was carried out : figure [ den1 ] .
the ward minimum variance agglomerative criterion was used , with equal weighting of the 8 nouns .
figure [ den2 ] shows a canonical representation of the dendrogram in figure [ den1 ] .
both trees are isomorphic to one another .
figure [ den2 ] is shown such that the sequence of agglomerations is portrayed from left to right ( and of course from bottom to top ) .
we say that figure [ den2 ] is a canonical representation of the dendrogram , implying that figure [ den1 ] is not in canonical form . in figure
[ den3 ] , the canonical representation has its non - terminal nodes labeled .
next , figure [ den4 ] shows a superimposed oriented binary rooted tree , on @xmath47 nodes , which is isomorphic to the dendrogram on @xmath48 terminal nodes .
this oriented binary tree is an inorder traversal of the dendrogram .
sibson s [ 1973 ] `` packed representation '' of a dendrogram uses just such an oriented binary rooted tree , in order to define a permutation representation of the dendrogram . from our example , the packed representation permutation can be read off as : @xmath49 : for any terminal node indexed by @xmath14 , with the exception of the rightmost which will always be @xmath48 , define @xmath50 as the rank at which the terminal node is first united with some terminal node to its right .
discussion of combinatorial properties of dendrograms , as related to such oriented binary rooted trees , and associated down - up and up - down permutations , can be found in murtagh [ 1984a ] .    , but now with successively _ later _ agglomerations always represented by _
right _ child node .
apart from the labels of the initial pairwise agglomerations , this is otherwise a unique representation of the dendrogram ( hence : `` existence '' and `` object '' can be interchanged ; so can `` disposition '' and `` fact '' ; and finally `` name '' and `` disposition '' ) . in the discussion we refer to this representation , with later agglomerations always parked to the right , as our canonical representation of the dendrogram.,width=529 ]    , with non - terminal nodes numbered in sequence .
these will form the nodes of the oriented binary tree .
we may consider one further node for completeness , 8 or @xmath51 , located at an arbitrary location in the upper right.,width=529 ]            finally , in figure [ den5 ] , we `` promote '' terminal node labels to the nodes of the oriented tree .
we will use exactly the procedure used above for defining a permutation representation of the oriented tree .
first the left terminal label is promoted to its non - terminal node .
next , the right terminal label is promoted as far up the tree as is necessary in order to find an unlabeled non - terminal node .
this procedure is carried out for all non - terminal labels , working in sequence from left to right ( i.e. , consistent with our canonical representation of the dendrogram ) .
the rightmost label is not shown : it is at an arbitrary location in the upper right hand side , with a tree arc oriented towards the top non - terminal node of the dendrogram , now labeled as `` motion '' .    in this section ,
we have specified a consistent procedure for labeling the nodes of an oriented tree , starting from the labels associated with the terminal nodes of a dendrogram .
we start therefore with embedded clusters , and end up with terms and directed links between these terms .
there is some non - uniqueness : any two labels associated with terminal nodes that are left and right child nodes of one non - terminal node can be interchanged .
this clearly leads to a different label promotion outcome .
our promotion procedure was motivated by the permutation representation of an oriented binary tree , as described above . here
too we do not claim uniqueness of permutation representation .
but we do claim optimality in the sense of parsimony , and well - definedness .    in the case of a multiway tree with very few distinct levels
, the promotion procedure becomes very simple , but continues to be non - unique .      in the previous subsection
, we discussed an algorithm which takes a hierarchical clustering , and hence a dendrogram , into an inorder tree traversal , and hence a permutation of the set of terms used .
the formal procedure discussed in the previous subsection suffers from non - uniqueness : alternative permutations could be defined .
this leads us to question the relationship of subsumption ( or direction in the oriented tree ) . in this section
we will develop another approach which is even more closely associated with the data that we are analyzing .
we have already seen that triangle properties between triplets of points , or data objects , are fundamental to ultrametricity and hence to tree representation .
a dendrogram , representing a hierarchical clustering , allows us to read off , for all triplets of points , either ( i ) isosceles triangles , with small base , or ( ii ) equilateral triangles , and ( iii ) no other triangle configuration .
the reason for the last condition is simply that non - isosceles , or isosceles with large base , triangles are incompatible with the ultrametric , or tree , metric .
we will leave aside for the present the equilateral triangle case .
firstly , it implies that all 3 points are _ ex aequo _ in the same cluster .
secondly , therefore we will treat them altogether as a concept cluster .
thirdly , the equilateral case does not arise in the example we will now explore .     and [ den3 ] , with clusters indicated by ellipses .
shown here are ellipses covering the clusters at nodes 7 , 6 , 5 , and 3.,width=529 ]    in figure [ den6 ] , cluster number 3 indicates the following isosceles triangle with small base : ( ( existence , object ) position ) .
our notation is : ( ( x , y ) z ) , such that triplet x , y , z has small base x , y , and the side lengths x , z and y , z are equal . this
is necessarily implied by relationships represented in figure [ den6 ] .
so , motivated by this triangle view of the cluster number 3 part of the dendrogram we will promote `` position '' to the cluster number 3 node .
similarly we will promote `` motion '' to the cluster number 5 node .
note the consistency of our perspective on the cluster number 3 and 5 nodes relative to how the associated terms here form an isosceles triangle with small base .
we will straight away generalize this definition . in any case of a node in the form of nodes 3 or 5 , where we have a 2-term left subtree , and a 1-term right subtree , where left and right are necessarily labeled in this way given the canonical representation of the dendrogram , then : _ the left subtree is dominated by the right subtree_.    we will next look at cluster number 6 ( remaining with figure [ den6 ] ) . as always for such trees , the node corresponding to this cluster has two subtrees , one to the left ( here : 3 ) and one to the right ( here : 5 ) .
since our dendrogram is in canonical form , any such node has a subtree with smallest non - terminal node level to the left ; and the subtree which was more recently formed in the sequence of agglomerations to the right .
based on either or both of these criteria which serve to define what are the left and right subtrees we define the ordering relationship : _ the left subtree is dominated by the right subtree_.    figure not available : see pdf version of paper at www.cs.rhul.ac.uk/home/fionn/papers/auto_onto.pdf    figure [ concepts ] summarizes the concept relations that we can derive in a similar way from any dendrogram .
figure [ aristontoex ] indicates how the concept relations , shown in figure [ concepts ] , are to be used .    firstly the term set is summarized , using our selection of terms . scaling to large data sets
is addressed in this way .    secondly , in our interactive implementation ( web address : + thames.cs.rhul.ac.uk/@xmath52dimitri/textmap ) , we allow the terms shown to continually move in a limited way , to get around the occlusion problem , and we also allow magnification of the display area for this same reason .
thirdly , terms other than those shown are highlighted when a cursor is passed over them .
next , double clicking on any term gives a ranked list of text segment names , ordered by frequency of occurrence by this term .
clicking on the text segment gives the actual text at the bottom of the display area .
are shown in decreasing size ( and in rainbow colors , from red ) , with other terms ( in all , 66 ) displayed with a dash , and all text segments ( in all , 24 ) represented by an asterisk .
the principal factor plane of a correspondence analysis ( based on the 24 text segments @xmath53 66 terms frequencies of occurence ) output is used.,width=529 ]      we proceed now to a third case study of this work , where we have a multiway hieararchy ( and not a binary hierarchy ) from the start .
we require a frequency of occurrence matrix which crosses the terms of interest with parts of a free text document . for the latter we could well take documentary segments like paragraphs .
oneill [ 2006 ] is a 660-word discussion of ubiquitous computing from the perspective of human computing interaction . with this short document we used individual lines ( as proxies for the sequence of sentences ) as the component parts of the document .
there were 65 lines .
this facilitates retrieval of a small segment of such a single document .
we chose this text to work with because it is a very small text ( a single text compared to the data used in section [ sect6 ] , and a far smaller text compared to that used in section [ secorientree ] ) .    based on a list of nouns and substantives furnished by the part - of - speech tagger ( schmid , 1994 ) ,
we focused on the following 30 nouns :    support = @xmath54 `` agents '' , `` algorithms '' , `` aspects '' , `` attempts '' , `` behaviours '' , `` concepts '' , `` criteria '' , `` disciplines '' , `` engineers '' , `` factors '' , `` goals '' , `` interactions '' , `` kinds '' , `` meanings '' , `` methods '' , `` models '' , `` notions '' , `` others '' , `` parts '' , `` people '' , `` perceptions '' , `` perspectives '' , `` principles '' , `` systems '' , `` techniques '' , `` terms '' , `` theories '' , `` tools '' , `` trusts '' , `` users '' @xmath55 .
this set of 30 terms was used to characterize through presence / absence the 65 successive lines of text , leading to correspondence analysis of the @xmath56 presence / absence matrix .
this yielded then the definition of the 30 terms in a factor space . in principle
the rank of this space ( taking account of the trivial first factor in correspondence analysis , relating to the centering of the cloud of points ) is min ( @xmath57 ) .
however , given the existence of zero - valued rows and/or columns , the actual rank was 25 .
therefore the full rank projection of the terms into the factor space gave rise to 25-dimensional vectors for each term , and these vectors are endowed with the euclidean metric .
define this set of 30 terms as the _ support _ of the document . based on their occurrences in the document
, we generated the following _ reduced _ version of the document , defined on this support , which consists of the following ordered set of 52 terms :    reduced document = `` goals '' `` techniques '' `` goals '' `` disciplines '' `` meanings '' `` terms '' `` others '' `` systems '' `` attempts '' `` parts '' `` trusts ''
`` trusts '' `` people '' `` concepts '' `` agents '' `` notions '' `` systems '' `` people '' `` kinds '' `` behaviours '' `` people '' `` factors '' `` behaviours '' `` perspectives '' `` goals '' `` perspectives '' `` principles '' `` aspects '' `` engineers '' `` tools '' `` goals '' `` perspectives '' `` methods '' `` techniques '' `` criteria '' `` criteria '' `` perspectives '' `` methods '' `` techniques '' `` principles '' `` concepts '' `` models '' `` theories '' `` goals '' `` tools '' `` techniques '' `` systems '' `` interactions '' `` interactions '' `` users '' `` perceptions '' `` algorithms ''    this reduced document is just the `` time series '' of the nouns of interest to us , as they are used in traversing the document from start to finish . each noun in the sequence of 52 nouns is represented by its 25-dimensional factor space vector .    out of 43 unique triplets , with self - distances removed , we found 31 to respect the ultrametric inequality , i.e.  72% .
our measure of ultrametricity of this document , based on the support used , was thus 0.72 .    for a concept hierarchy
we need an overall fit to the data . using the euclidean space perspective on the data , furnished by correspondence analysis
, we can easily define a terms @xmath53 terms distance matrix ; and then hierarchically cluster that .
consistent with our analysis we recode all these distances , using the cvnc mapping onto @xmath58 for unique pairs of terms .
now approximating a global ultrametric from below , achieved by the single linkage agglomerative hierarchical clustering method ( and this best fit from below , termed the subdominant or maximal inferior ultrametric , is optimal ) , and an approximation from above , achieved by the complete linkage agglomerative hierarchical clustering method ( and this best fit from above , termed a minimal superior ultrametric , is non - unique and hence is one of a number of best fits from above ) , will be identical if the data is fully ultrametric - embeddable .
if we had an ultrametricity coefficient equal to 1  we found it to be 0.72 for this data  then it would not matter what agglomerative hierarchical clustering algorithm ( among the usual lance - williams methods ) that we select .
in fact , we found , with an ultrametricity coefficient equal to 0.72 , that the single and complete linkage methods gave an identical result .
this result is shown in figure [ fig1 ] .
a convenient label promotion procedure to apply here is first to re - represent the terminal labels from left to right as : @xmath54 `` users '' , `` trusts '' , @xmath59 , `` agents '' , `` algorithms '' @xmath55 ; @xmath54 `` goals '' , `` perspectives '' @xmath55 ; @xmath54 `` tools '' @xmath55 ; @xmath54 `` techniques '' @xmath55 ; and @xmath54 `` methods '' @xmath55 .
this is the canonical form , with ordering of left and right subtrees now extended to all subtrees .
next , we must in fairness take the nodes at level 2 as being _
ex aequo _ , @xmath54 `` tools '' @xmath55 ; @xmath54 `` techniques '' @xmath55 ; and @xmath54 `` methods '' @xmath55 . similarly at level 1 , we also have two clusters that are _ ex aequo _ :
@xmath54 `` users '' , `` trusts '' , @xmath59 , `` agents '' , `` algorithms '' @xmath55 ; and @xmath54 `` goals '' , `` perspectives '' @xmath55 .    figure not available : see pdf version of paper at www.cs.rhul.ac.uk/home/fionn/papers/auto_onto.pdf    figure [ ubiq ] shows our resulting scheme where level 1 clusters dominate level 2 clusters .
this provides our ontology .
the granularity of this one document is , as mentioned above , line - based , and there are 65 lines in all . hence retrieval of one or more of these document snippets is supported , and the ontology is based on a 30-noun document support .
having first appraised text collections in terms of their local hierarchical structure , we then proceeded in this work to show how this new methodology could be employed for a wide range of tasks that include :    * finding salient pairs and triplets of terms , which are not necessarily in sequence ; * permitting us to consider any given text as a whole with all pairwise relationships between terms , or alternatively as a time series with relationships restricted to terms that are successive in sequence ; * passing seamlessly from the exploration of local hierarchical structure to global hierarchical structure ; * especially when global hierarchical structure is manifest , being able to use any of a wide range of agglomerative clustering criteria to furnish the same resultant hierarchy ; * determining a hierarchy of concepts from the embedded , partially ordered subsets provided by a hierarchical clustering ; * obtaining unique results when given a 2-way hierarchical clustering tree , and then readily generalizing this to the practical case of multiway trees ; * exemplifying an efficient and an effective textual data processing pipeline ; and * through the measurement of local hierarchical structure , having available an approach to validating the appropriateness of any data for this data analysis pipeline .    by analysis of text through local hierarchical relationships between terms we determine extensive internal textual structure , without being stifled by the more traditional approach of fitting some global structure , such as a hierarchy , to the text .
a ( local ) hierarchical structure is a powerful one : it includes peer as well as subsumption types of relationships .
we stress that we can find very pronounced hierarchical structures of this sort if we encode the text is novel ways .
an example is to start with a euclidean spatial embedding of the terms and documents ( or segments of a document ) , which is quite traditional ; and then look at interrelationships between terms using `` relatively close / similar '' versus `` relatively distant / new '' ( and this alone can be shown to have metric properties ) .
another example of an encoding - related strategy is not to take into consideration all interrelationships between terms , but only between successive terms , and thereby view the text as a particular type of time series .
user interactivity with the system is to select the terms of interest ( people s personal names , industrial product names , location or venue names , etc . ) .
the interrelationships between these terms are then explored through their local hierarchical links .
our general application targeted is , as stated in murtagh et al .  [
2003 ] , to have readily available a self - description of data , as a basis for visually - based interactive and responsive querying of , retrieval from , and navigation of data collections .
this work was carried out in the context of the european union sixth framework project , `` ws - talk , web services communicating in the language of their community '' , 20042006 .
pedro contreras and dimitri zervas contributed to this work .
the textmap demonstrator was developed by dimitri zervas .
: :    abou assali , a. and zanghi , h. 2006 . automated metadata hierarchy    derivation
. in _ proc .
ieee ictta06 _
( damascus , syria ) .
: :    ahmad , k. and gillam , l. ( 2005 ) .
automatic ontology extraction from    unstructured texts . in _
odbase 2005_. : :    benzcri , j.p .
_ lanalyse des donnes , tome i taxinomie , tome    ii correspondances _ , 2nd ed .
( dunod , paris ) .
: :    chuang shui - lung and chien lee - feng ( 2005 ) .
taxonomy generation for    text segments : a practical web - based approach , acm transactions on    information systems .
23 , 363396 .
: :    de soete , g. ( 1986 ) . a least squares algorithm for fitting an    ultrametric tree to a dissimilarity matrix . _
pattern recognition    letters _ , 2 , 133137 . : :    denny , m. ( 2004 ) .
ontology tools survey , revisited , at +    http://www.xml.com/pub/a/2004/07/14/onto.html , july 2004 .
: :    doyle , l.b .
semantic road maps for literature searches .
_ journal of the acm_. 8 , 553578 .
: :    ganesan , p. , garcia - molina , h. and widom , j. ( 2003 ) . exploiting    hierarchical domain structure to compute similarity
_ acm transactions    on information systems_. 21 , 6493 .
: :    gmez - prez , a. , fernndez - lpez , m. and corcho , o. ( 2004 ) .    _ ontological engineering ( with examples from the areas of knowledge    management , e - commerce and the semantic web ) _ ( springer , berlin ) . : :    gruber , t. ( 2001 ) . what is an ontology ?
,    http://www-ksl.stanford.edu/kst/what-is-an-ontology.html , sept .
: :    janowitz , m.f .
cluster analysis based on abstract posets ,    preprint . also presentation , enst de bretagne , 30 oct .
2004 ; dimacs , 9    mar .
2005 ; and sfc , 31 may 2005 .
: :    maedche , a. ( 2006 ) . _ ontology learning for the semantic web _
( kluwer ,    dordrecht ) .
: :    murtagh , f. ( 1984a ) . counting dendrograms : a survey . _ discrete applied    mathematics _ , 7 , 191199 .
: :    murtagh , f. ( 1984b ) . structures of hierarchic clusterings :    implications for information retrieval and for multivariate data    analysis . _ information processing and management _ , 20 , 611617 .
: :    murtagh , f. ( 2004 ) . on ultrametricity , data coding , and computation ,    _ journal of classification
, 21 , 167184 .
: :    murtagh , f. ( 2005a ) . identifying the ultrametricity of time series ,    _
european physical journal b _ , 43 , 573579 .
: :    murtagh , f. ( 2005b ) .
_ correspondence analysis and data coding with r    and java _
( chapman & hall / crc ) .
: :    murtagh , f. ( 2006a ) .
ultrametricity in data : identifying and    exploiting local and global hierarchical structure ,    arxiv : math.st/0605555v1 19 may 2006 .
: :    murtagh , f. ( 2006b ) .
the haar wavelet transform of a dendrogram , +    arxiv : cs.ir/0608107v1 28 aug 2006 , forthcoming in _ journal of    classification_. : :    murtagh , f. ( 2006c ) .
symbolic dynamics in text : application to    automated construction of concept hierarchies , _ festschrift for edwin    diday _ , in press .
available at : www.cs.rhul.ac.uk/home/fionn/papers : :    murtagh .
f. ( 2006d ) .
visual user interfaces , interactive maps ,    references , theses ,    http://astro.u-strasbg.fr/@xmath52fmurtagh/inform : :    murtagh , f. , taskaya , t. , contreras , p. , mothe , j. and englmeier , k.    ( 2003 ) .
interactive visual user interfaces : a survey , _ artificial    intelligence review _ , 19 , 263283 . : :    oneill , e. ( 2006 ) . understanding ubiquitous computing : a view from    hci , in discussion following r. milner , ubiquitous computing : how will    we understand it ? , _ computer journal _
, 49 , 390399 .
: :    schmid , h. ( 1994 ) .
probabilistic part - of - speech tagging using decision    trees . in _ proc .
new methods in language processing _ ( see    treetagger site ) . : :    treetagger , www.ims.uni-stuttgart.de/projekte/corplex/treetagger/    decisiontreetagger.html : :    sibson , r. ( 1973 ) .
slink : an optimally efficient algorithm for the    single - link cluster method , _ computer journal _ , 16 , 3034 .
: :    wache , h. , vgele , t. , visser , u. , stuckenschmidt , h. , schuster , g. ,    neumann , h. and hbner , s. ( 2001 ) .
ontology - based integration of    information  a survey of existing approaches . in _ proceedings of the    ijcai-01 workshop : ontologies and information sharing_.