Publication:
Combining heterogeneous inputs for the development of adaptive and multimodal interaction systems

Loading...
Thumbnail Image
Identifiers
Publication date
2013-12
Defense date
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Universidad de Salamanca
Impact
Google Scholar
Export
Research Projects
Organizational Units
Journal Issue
Abstract
In this paper we present a novel framework for the integration of visual sensor networks and speech-based interfaces. Our proposal follows the standard reference architecture in fusion systems (JDL), and combines different techniques related to Artificial Intelligence, Natural Language Processing and User Modeling to provide an enhanced interaction with their users. Firstly, the framework integrates a Cooperative Surveillance Multi-Agent System (CS-MAS), which includes several types of autonomous agents working in a coalition to track and make inferences on the positions of the targets. Secondly, enhanced conversational agents facilitate human-computer interaction by means of speech interaction. Thirdly, a statistical methodology allows modeling the user conversational behavior, which is learned from an initial corpus and improved with the knowledge acquired from the successive interactions. A technique is proposed to facilitate the multimodal fusion of these information sources and consider the result for the decision of the next system action.
Description
Keywords
Software agents, Multimodal fusion, Visual sensor networks, Surveillance applications, Spoken interaction, Conversational Agents, User Modeling, Dialog Management
Bibliographic citation
Advances in Distributed Computing And Artificial Intelligence Journal (2013). 2(6), 37-53.