Abstract: In this paper, a system for object-based semi-automatic indexing and retrieval of natural images is introduced. Three important concepts underpin the proposed system: a new strategy to fuse different low-level content descriptions; a learning technique involving user relevance feedback; and a novel object based model to link semantic terms and visual objects. To achieve high accuracy in the retrieval and subsequent annotation processes several low-level image primitives are combined in a suitable multifeatures space. This space is modelled in a structured way exploiting both low-level features and spatial contextual relations of image blocks. Support vector machines are used to learn from gathered information through relevance feedback. An adaptive convolution kernel is defined to handle the proposed structured multifeature space. The positive definite property of the introduced kernel is proven, as essential condition for uniqueness and optimality of the convex optimization in support vector machines. The proposed system has been thoroughly evaluated and selected results are reported in this paper

Abstract: Voluminous medical images are generated daily. They are critical assets for medical diagnosis, research, and teaching. To facilitate automatic indexing and retrieval of large medical-image databases, both images and associated texts are indexed using medical concepts from the Unified Medical Language System (UMLS) meta-thesaurus. We propose a structured learning framework based on support vector machines to facilitate modular design and learning of medical semantics from images. We present two complementary visual indexing approaches within this framework: a global indexing to access image modality and a local indexing to access semantic local features. Two fusion approaches are developed to improve textual retrieval using the UMLS-based image indexing. First, a simple fusion of the textual and visual retrieval approaches is proposed, improving significantly the retrieval results of both text and image retrieval. Second, a visual modality filtering is designed to remove visually aberrant images according to the query modality concept(s). Using the ImageCLEFmed database, we demonstrate the effectiveness of our framework which is superior when compared with the automatic runs evaluated in 2005 on the same medical-image retrieval task.

Abstract: Recently, communication, digital music creation, and computer storage technology has led to the dynamic increasing of online music repositories in both number and size, where automatic content-based indexing is critical for users to identify possible favorite music pieces. Timbre recognition is one of the important subtasks for such an indexing purpose. Lots of research has been carried out in exploring new sound features to describe the characteristics of a musical sound. The moving picture expert group (MPEG) provides a standard set of multimedia features, including low level acoustical features based on latest research in this area. This paper introduces our newly designed temporal features used for automatic indexing of musical sounds and evaluates them with MPEG7 descriptors, and other popular features.

Abstract: This paper presents a novel surveillance video indexing and retrieval system based on object features similarity measurement. The system firstly extracts moving objects from the videos by an efficient motion segmentation method. The fundamental features of each moving object are then extracted and indexed into the database. During retrieval, the system matches the query with the features indexed in the database without re-processing the videos. Video clips which contain the objects with sufficiently high relevance scores are then returned. The novelty of the system includes: 1. A real-time automatic indexing methodology achieved by a fast motion segmentation, such that the system is able to perform on-the-fly indexing on video sources; and 2. an object-based retrieval system with fundamental features matching approach, which allows user to specify the query by providing an example image or even a sketch of the desired objects. Such an approach can search the desired video clips in a more convenient and unambiguous way comparing with traditional text-based matching.

Abstract: An on-line document retrieval system is described which combines a data base management system with automatic processing of natural language queries and abstracts. Data consists of an abstract, from which index terms are automatically extracted, along with bibliographic and descriptive information. The data base management system is used to store bibliographic and descriptive information, providing direct access to documents with specified bibliographic or descriptor items. Methods originally developed in the SMART project are used for abstract analysis: stemming algorithm, cosine function for query-document comparisons, ranked output, and clustered document collection. Searches are entered and performed on-line, with output consisting of document abstracts ranked in decreasing order of similarity with the query. Additional facilities include off-line searches, SDI, and display of data base statistics. Future plans and improvements are also discussed.

Abstract: This paper presents annotations needed for handwritten archive document retrieval by content. We propose two complementary ways of producing these annotations: automatically by using document image analysis and collectively by using the Internet and manual input by users. A platform for managing these annotations is presented as well as examples of automatic annotations on civil status registers, military forms (tested on 165,000 pages) and naturalization decrees, using a generic method for structured document recognition and handwriting recognition on names. Examples of collective annotations built on automatic annotations are also given. This platform is already open to the public in the reading room of the new building of the Archives departementales des Yvelines and on the Internet. About 1,450,000 images of civil status registers are available for collective annotation as well as 105,000 pages of military forms with automatic annotation of handwritten names.

Abstract: In most natural language processing systems there is no representation of the semantic knowledge of lexical units, but just subcategorization frames, selection restrictions and links to other paradigmatically-related lexical units. Some NLP systems, e.g. machine translation or dialogue-based systems, attempt to "understand" the input text by translating it into some kind of formal language-independent representation; this approach requires a knowledge base with conceptual representations which reflect the structure of human beings' cognitive system. Even those systems in which surface semantics could be sufficient (e.g. automatic indexing or information extraction), the construction of a robust knowledge base guarantees its use in most natural language processing tasks, consolidating thus the concept of resource reuse. The objective of this paper is to highlight the advantages of storing conceptual meaning representations, and more particularly those in FunGramKB, instead of describing lexical meaning via semantic relations between lexical units.

Abstract: Musical instrument sounds can be classified in various ways, depending on the instrument or articulation classification. This paper presents a number of possible generalizations of musical instruments sounds classification which can be used to construct different hierarchical decision attributes. Each decision attribute will lead us to a new classifier and the same to a different system for automatic indexing of music by instrument sounds and their generalizations. Values of a decision attribute and their generalizations are used to construct atomic queries of a query language built for retrieving musical objects from MIR database (see http://www.mir.uncc.edu). When query fails, the cooperative strategy will try to find its lowest generalization which does not fail, taking into consideration all available hierarchical attributes. Thus, the music object representing most similar object in the database is returned as the query answer. This paper evaluates only two hierarchical attributes upon the same dataset which contains 2628 distinct musical samples of 102 instruments from McGill University Master Samples (MUMS) CD collection.

Abstract: With the fast booming of online music repositories, there is a need for content-based automatic indexing which will help users to find their favorite music objects in real time. Recently, numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of these methods can be successfully applied to polyphonic sounds. Identification of music instruments in polyphonic sounds is still difficult and challenging, especially when harmonic partials are overlapping with each other. This has stimulated the research on music sound separation and new features development for content-based automatic music information retrieval. Our goal is to build a cooperative query answering system (QAS), for a musical database, retrieving from it all objects satisfying queries like "find all musical pieces in pentatonic scale with a viola and piano where viola is playing for minimum 20 seconds and piano for minimum 10 seconds". We use the database of musical sounds, containing almost 4000 sounds taken from the MUMs (McGill University Master Samples), as a vehicle to construct several classifiers for automatic instrument recognition. Classifiers showing the best performance are adopted for automatic indexing of musical pieces by instruments. Our musical database has an FS-tree (Frame Segment Tree) structure representation. The cooperativeness of QAS is driven by several hierarchical structures used for classifying musical instruments.

Abstract: Most text classification techniques assume that manually labeled documents (corpora) can be easily obtained while learning text classifiers However, labeled training documents are sometimes unavailable or inadequate even if they are available The goal of this article is to present a self-learned approach to extract high-quality training documents from the Web when the required manually labeled documents are unavailable or of poor quality To learn a text classifier automatically, we need only a set of user-defined categories and some highly related keywords Extensive experiments are conducted to evaluate the performance of the proposed approach using the test set from the Reuters-21578 news data set The experiments show that very promising results can be achieved only by using automatically extracted documents from the Web © 2007 Wiley Periodicals, Inc

Abstract: This paper describes a novel browsing paradigm, taking benefit of the various types of links (e.g. thematic, temporal, references, etc.) that can be automatically built between multimedia documents. This browsing paradigm can help eliciting multimedia archives' hidden structures or expanding search results to related media. The paper intend to present a novel model for browsing any kind of multimedia archives and further focuses on an archive of meetings recordings, in order to illustrate the advantage of our method to perform cross-meetings and in general cross-documents browsing. First of all, the structure of meeting datasets is presented, describing in particular the media implied, the annotations used for cross-document linking and the major mining techniques integrated in this work. Then, the paper presents at a glance the visual browser we developed that combines searching and browsing by links. Further, the performances of the actual system are discussed, i.e. the automatic indexing and linking processes for the two different meeting corpora, as well as the access and browsing performances. Finally, the paper presents the major unsolved issues and our perspectives for future works.

Abstract: Large quantities of historical newspapers are being digitized and OCRd. We describe a framework for processing the OCRd text to identify articles and extract metadata for them. We describe the article schema and provide examples of features that facilitate automatic indexing of them. For this processing, we employ lexical semantics, structural models, and community content. Furthermore, we describe visualization and summarization techniques that can be used to present the extracted events.

Abstract: Synonymy has been of high importance in information retrieval and automatic indexing. Recently, in the view of special needs for domain ontology building and maintenance, the problem returns with a higher demand. In the presented paper, we present a novel text mining approach to discovering synonyms or close meaning terms. The offered measures of closeness of terms (or their contexts) are expressed by means of data mining notions; namely, frequent termsets and association rules. The measures can be calculated by using data mining techniques, such as the well known Apriori algorithm. The approach is domain-independent and large-scale. It is, however, restricted to the recognition of parts of speech. In that sense the approach is language dependent, up to the language dependency of the parts of speech tagging process. The experimental results obtained with the approach are presented.

Abstract: This paper deals with spatial Information Extraction (IE) and Retrieval (IR) in Digital Libraries environments. The proposed approach (implemented within PIV(1) prototype) is based on a linguistic and semantic analysis of digital corpora and free text queries. First, we present requirements and a methodology of semantic annotation for automatic indexing and geo-referencing of text documents. Then we report on a case study where the spatial-based IR process is evaluated and compared to classical (statistical-based) IR approaches using first pure spatial queries and then more general ones dealing with both spatial and thematic scopes. The main result in these first experiments shows that combining a spatial approach with a classical (statistical-based) IR one improves in a significant way retrieval accuracy, namely in the case of general queries.

Abstract: This paper proposes a neural network based approach for emotion-based textile indexing. Generally, the human emotion can be affected by some physical features such as color, texture, pattern, and so on. In the previous work, we investigated the correlation between the human emotion and color or texture. Here, we aim at investigating the correlation between the emotion and pattern, and developing the textile indexing system using the pattern information. Therefore, the survey is first conducted to investigate the correlation between the emotion and the pattern. The result shows that a human emotion is deeply affected by the certain pattern. Based on that result, an automatic indexing system is developed. The proposed system is composed of feature extraction and classification. To describe the pattern information in the textiles, the wavelet transform is used. And the neural network is used as the classifier. To assess the validity of the proposed method, it was applied to recognize the human emotions in 100 textiles, and then our system produced the accuracy of 90%. This result confirmed that our system has the potential to be applied for various applications such as textile industry and e-business.

Abstract: This paper proposes a neural network based approach for emotion based textile indexing. Generally, the human emotion can be affected by some physical features such as color, texture, pattern, and so on. In the previous work, we investigated the correlation between the human emotion and color or texture. Here, we aim at investigating the correlation between the emotion and pattern, and developing the textile indexing system using the pattern information. Therefore, the survey is first conducted to investigate the correlation between the emotion and the pattern. The result shows that a human emotion is deeply affected by the certain pattern. Based on that result, an automatic indexing system is developed. The proposed system is composed of feature extraction and classification. To describe the pattern information in the textiles, the wavelet transform is used. And the neural network is used as the classifier. To assess the validity of the proposed method, it was applied to recognize the human emotions in 100 textiles, and then our system produced the accuracy of 90%. This result confirmed that our system has the potential to be applied for various applications such as textile industry and e-business.

Abstract: The use of thesaurus-based indexing is a common approach for increasing the performance of document retrieval. With the growing amount of documents available, manual indexing is not a feasible option. Statistical methods for automated document indexing are an attractive alternative. We argue that the quality of the thesaurus used as a basis for indexing in regard to its ability to adequately cover the contents to be indexed is of crucial importance inautomatic indexing because there is no human in the loop that can spot and avoid indexing errors. We propose a method for thesaurus evaluation that is based on a combination of statistical measures and appropriate visualization techniques that supports the detection of potential problems in a thesaurus. We describe this method and show its application in the context of two automatic indexing tasks. The examples show that the methods indeed eases the detection and correction of errors leading to a better indexing result. Please refer to http://www.kaiec.org for high resolution media of all figures used in this paper, as well as an animated presentation of the interactive tool.

Abstract: Most of information retrieval (IR) approaches relies on the hypothesis that keywords extracted from a document are sufficient to evaluate the relevance of that document with respect to the query. Such an approach may insufficiently lay bare the semantic contents of the documents. In addition to keywords, automatic indexing methods need external knowledge such as thesauri and ontologies for improving the representation of documents or for expanding queries to related keywords. Moreover, ontologies may be combined with a view of for estimating the relevance of documents, the “proximity” between words, or for expressing flexible queries. In this chapter, we survey several recent approaches. Then, two types of methods are discussed in detail. The first one uses a symbolic pattern matching approach, which is based on possibilistic ontologies (where qualitative necessity and possibility degrees estimate to what extent two terms refer to the same thing). The second type of approaches projects fuzzy set representations of queries and documents on a classical ontology, and compare these projections for rank ordering the documents according to a retrieval status value.

Abstract: The growing number of resources to be indexed in the catalogue of online health resources in French (CISMeF) calls for curating strategies involving automatic indexing tools while maintaining the catalogue's high indexing quality standards. Objective: To develop a simple automatic tool that retrieves MeSH descriptors from documents titles. Methods: In parallel to research on advanced indexing methods, a bag-of-words tool was developed for timely inclusion in CISMeF's maintenance system. An evaluation was carried out on a corpus of 99 documents. The indexing sets retrieved by the automatic tool were compared to manual indexing based on the title and on the full text of resources. Results: 58% of the major main headings were retrieved by the bag-of-words algorithm and the precision on main heading retrieval was 69%. Conclusion: Bag-of-words indexing has effectively been used on selected resources to be included in CISMeF since August 2006. Meanwhile, on going work aims at improving the current version of the tool.

Abstract: With the fast booming of online music repositories, there are increasing needs for content-based Automatic Indexing to help users find their favorite music objects. Music instrument recognition is one of the main subtasks. Recently, numerous successful approaches on musical data feature extraction and selection have been proposed for instrument recognition in monophonic sounds. Unfortunately, none of those algorithms can be successfully applied to polyphonic sounds. Thus, identification of music instruments in polyphonic sounds is still difficult and challenging, especially when harmonic partials are overlapping with each other. This has stimulated the research on music sound separation and new features development for content-based automatic music information retrieval. Based on recent research results in sound classification of monophonic sounds and studies in speech recognition, Moving Picture Experts Group (MPEG) standardized a set of features of the digital audio content data for the purpose of interpretation of the information meaning for audio signal. Most of them are in a form of large matrix or a vector of large size, which are not suitable for traditional data mining algorithms; while other features in a smaller size are not sufficient for instrument recognition in polyphonic sounds. Therefore, these acoustical features themselves alone cannot be successfully applied to classification of polyphonic sounds. However, these features contain critical information, which implies music instruments' signatures. The ultimate goal of this thesis is to build a flexible query answering system, for a musical database, retrieving from it all objects satisfying queries like "find all musical pieces in pentatonic scale with a viola and piano where viola is playing for minimum 20 seconds and piano for minimum 10 seconds". To achieve that, first of all a database of sounds containing musical instruments allowed in queries has to be built. This database is already built as a part of the music information retrieval system, called MIRAI, and it already contains about 4000 sounds taken from the MUMs (McGill University Master Samples). The descriptions of these sounds are in terms of standard musical features which definitions are provided by MPEG7, in terms of other features used earlier in a similar research, and new features proposed in this thesis. All these features are implemented and tested for their correctness. The database of musical sounds is used as a vehicle to construct several classifiers for automatic instrument recognition. In this thesis we limit our investigations to classifiers provided by WEKA and RSES (Rough Sets Exploration System). Their performance is compared against the performance of similar classifiers constructed from the same database projected to MPEG7 type features only. The main problem facing this thesis is not only the construction of the proper and sufficient set of features needed to represent musical sounds which guarantees that their descriptions can differentiate them but also a mechanism of splitting multiple instruments played simultaneously in musical sounds. For checking the performance of classifiers 3-cross or/and 10-cross validation, and bootstrap procedures are used. The classifiers showing the best performance are adopted for automatic indexing of musical pieces by instruments. Each musical piece is seen as a segmented object in the musical database with segments showing when each relevant instrument starts and ends playing. This way the musical database can be represented as an FS-tree (Frame Segment Tree) structure. The query answering system should be seen as the interface to the FS-tree representation of the musical database. The flexibility of the query answering system is based on the hierarchical structure representing all musical instruments. When a query fails, it is generalized and checked for success by the query answering system. The construction of the Flexible Query Answering System requires building classifiers, for automatic indexing of music by instruments, based on the training database where instruments names are replaced by their generalizations. Sound Separation is restricted to isolating the harmonic polyphonic sounds, where stable pitches are predictable. Previous research of different aspects on music instruments identification is reviewed by the order of sound processing. The complete process of answering user queries will involve the following functionalities: segmenting a music piece into groups of frames, estimating a pre-dominant pitch and isolating/subtracting the sound by matching its harmonic features with the feature database, repeating the process until only noise remains, retrieving features from the resultant monophonic sounds, performing classification and storing the labels together with the music piece in a form of the FS tree into an indexed database. This work has implications for research in blind harmonic sound separation, fundamental frequency estimation, timbre identification, pattern recognition, classification, music annotation, and collaborative query answering.

Abstract: This paper presents a novel hybrid method for content based visual information retrieval (CBIR) that combines shape analysis of objects in image with their automatic indexing by textual descriptions. The principal goal of proposed method is the applying semantic Web approaches for visual information description in systems which use the low-level image characteristics. In the proposed method the user-oriented textual queries are converted to image characteristics which are used for visual information seeking and matching analysis. A decision about similarity between a retrieved image and user queries is taken by computing the shape convergence star field or two-segment turning functions combining them with matching of ontological annotations of objects in image providing in this way the machine-understandable semantics. For analysis of proposed method the image retrieval IRONS (Image Retrieval by Ontological Description of Shapes) system has been designed and evaluated in some specific image-restricted domains

Abstract: A structured document storage and management technique utilizes a generic document model tree, a symbol conversion module and an encoded vector set to store structured documents. The generic document model tree represents a structured document model and contains one or more structured document nodes without storing node data unique to any particular structured document. The symbol conversion module contains untagged data associated with the one or more structured document nodes, and representing node data for particular structured documents. The symbol conversion module also maintains a value code in association with each untagged data element. The encoded vector set includes one or more encoded vectors corresponding to the one or more structured document nodes having associated untagged data. Each encoded vector contains one of the value codes at an index position that corresponds to a particular structured document. The disclosed technique allows structured documents to be efficiently stored, organized, and searched.

Abstract: The shift from paper to electronic documents has caused the curation of information sources in large electronic databases to become more generalized. In the biomedical domain, continuing efforts aim at refining indexing tools to assist with the update and maintenance of databases such as MEDLINE®. In this paper, we evaluate two statistical methods of producing MeSH® indexing recommendations for the genetics literature, including recommendations involving subheadings, which is a novel application for the methods. We show that a generic representation of the documents yields both better precision and recall. We also find that a domain-specific representation of the documents can contribute to enhancing recall.

Abstract: In this paper, we describe the SemanticVox project. SemanticVox aims at providing a real link between speech transcription technologies from Vecsys [8] based on LIMSI research [9] and multimedia documents analysis and retrieval technologies from the Multilingual Multimedia Knowledge Engineering Laboratory (LIC2M) of the CEA-LIST [1]. The first application of the project is a cross-lingual automatic video indexing and retrieval system based on speech transcription and video analysis. The two main novelties of the system are: (i) its ability to manage multilingual queries and documents; (ii) its innovative ranking to sort relevant documents based on a deep analysis of the syntax and semantic of the query.A video of this demonstration is available at http://www.eeng.dcu.ie/~hlborgne/semanticvox.wmv

Abstract: The high volume of digital music recordings in the internet repositories has brought a tremendous need for a cooperative recommendation system to help users to find their favorite music pieces. Music instrument identification is one of the important subtasks of a content-based automatic indexing, for which authors developed novel new temporal features and built a multi-hierarchical decision system S with all the low-level MPEG7 descriptors as well as other popular descriptors for describing music sound objects. The decision attributes in S are hierarchical and they include Hornbostel-Sachs classification and generalization by articulation. The information richness hidden in these descriptors has strong implication on the confidence of classifiers built from S. Rule-based classifiers give us approximate definitions of values of decision attributes and they are used as a tool by content-based Automatic Indexing Systems (AIS). Hierarchical decision attributes allow us to have the indexing done on different granularity levels of classes of music instruments. We can identify not only the instruments playing in a given music piece but also classes of instruments if the instrument level identification fails. The quality of AIS can be verified using precision and recall based on two interpretations: user and system-based [16]. AIS engine follows system-based interpretation.

Abstract: AMTEx is a medical document indexing method, specifically designed for the automatic indexing of documents in large medical collections, such as MEDLINE, the premier bibliographic database of the U.S. National Library of Medicine (NLM). AMTEx combines MeSH, the terminological thesaurus resource of NLM, with a wellestablished method for term extraction, the C/NC-value method. The performance evaluation of two AMTEx configurations is measured against the current state-of-theart, the MMTx method in indexing and retrieval tasks in three experiments. In the first, a subset of MEDLINE (PMC) full document corpus was used for the indexing task. In the second and third, a subset of MEDLINE (OHSUMED) abstracts was used for indexing and retrieval respectively. The experimental results demonstrate that AMTEx achieves better precision in all tasks, in 50-20% of the processing time compared to MMTx.

Abstract: This paper brings two contributions in relation with the semantic heterogeneous (documents composed of texts and images) information retrieval: (1) A new context-based semantic distance measure for textual data, and (2) an IR system providing a conceptual and an automatic indexing of documents by considering their heterogeneous content using a domain specific ontology. The proposed semantic distance measure is used in order to automatically fuzzify our domain ontology. The two proposals are evaluated and very interesting results were obtained. Using our semantic distance measure, we obtained a correlation ratio of 0.89 with human judgments on a set of words pairs which led our measure to outperform all the other measures. Preliminary combination results obtained on a specialized corpus of web pages are also reported.

Abstract: Former layouts contain much of the know-how of architects. A generic and automatic way to formalize this know-how in order to use it by a computer would save a lot of effort and money. However, there seems to be no such way. The only access to the Know-how are th.e layouts themselves. Developing a generic software tool to reuse former layouts you cannot consider every part of the architectural domain or things like personal style. Tools used today only consider small parts of the architectural domain. Any personal style is ignored Isn't it possible to build a basic tool which is adjusted by the content of the former layouts, but maybe extended inclemently by modeling as much of the domain as desirable? This paper will describe a reuse tool to perform this task focusing on topological and geometrical binary relations.

Abstract: Evaluation of automatic indexing is a common practice. Whenever new algorithms are created to generate semantic structures, it is necessary for their designers to demonstrate how better they are in the particular context for which they have been developed. However, the evaluation depends on the availability of a result of reference. Should such a reference be missing, evaluation is replaced by confrontation. There the structures themselves have to be compared instead of shared metrics, sometimes without even the option to choose the best one. Our work takes place in this context. We present a method for human-driven confrontation of specialized semantic structures. Our aim is to improve the user's ability to use another person's work to enhance his.