Publication: Un marco de trabajo para la evaluación predictiva de sistemas de notificación en el campo de la aviónica
Loading...
Identifiers
Files
Publication date
2006
Defense date
2006-12-12
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
Es inaudito encontrar un sistema de cualquier tipo basado en computadores en el que el humano no interactúe con él. Incluso aunque se trate de una comunicación exclusivamente entre dos computadores, normalmente se incluye una interfaz que permita solucionar el enlace entre máquinas cuando aparece algún problema [Mulligan et al. 1991].
El usuario, como ser humano, necesita información, lo que se considera una de las pocas constantes en nuestro mundo, independientemente de la cultura a la que pertenezca o en el instante de tiempo en el que transcurra su actividad. Esta necesidad se combina con la ocupación del usuario en otras tareas que requieren su atención, de forma que hay al menos dos focos de atención que compiten por el bien más preciado de cualquier interfaz humano-computador: la atención del usuario.
La existencia de diferentes escenarios, en las que el usuario espera resultados diferentes y consecuencias distintas ante la existencia de un error controla la aparición de distintos tipos de aplicaciones y dispositivos para soportar las demandas del usuario en condiciones de multitarea. En estos sistemas, conocidos como Sistemas de Notificación, las interfaces están diseñadas como un medio para acceder a información valiosa para el usuario de una forma efectiva y eficiente sin introducir interrupciones no deseadas en la ejecución de la tarea principal. Pueden encontrarse numerosas implementaciones de Sistemas de Notificación en una gran variedad de plataformas. Quizás en los sistemas de sobremesa son más fácilmente identificables: servicios de mensajería instantánea, aplicaciones que muestran el estado de un sistema, servicios de noticias e indicadores de cantidades almacenadas. Otros ejemplos familiares pueden encontrarse en procesos industriales y tecnológicos como la cadena danzante de Weiser como representación del tráfico de una red [Weiser y Brown 1996], los sistemas de información embarcados en vehículos, los sistemas incorporados en los vehículos para proporcionar información de los sistemas, los sistemas que proporcionan información sobre el entorno, los sistemas multi-monitor que indican los excesos de rango de un sistema y las cabinas de los aviones modernos.
De otro lado, un sistema se considera crítico cuando un único fallo debido al sistema, al usuario o al modo en el que el usuario realiza las tareas para las que el sistema se diseñó, puede producir daños graves en el sistema, en su entorno o, aún más grave, en los usuarios del sistema. Ejemplos de este tipo de sistemas pueden encontrarse en los aviones, correspondiéndose con sistemas manejados por usuarios muy especializados.
Genéricamente hablando, la evaluación de usabilidad incluye un amplio rango de métodos y técnicas usados por los evaluadores para evaluar, cualitativa o cuantitativamente, distintos aspectos relacionados con la usabilidad. Como soporte al diseño, la evaluación debe permitir, en primer lugar, la verificación de las decisiones de diseño y en segundo lugar elegir entre varias alternativas.
Aunque se le conceda gran importancia, la evaluación de cualquier sistema está poco implantada como un proceso más en el ciclo de vida de un producto, ya que se suele considerar un proceso costoso, en tiempo y recursos, y que no añade ningún valor al producto.
La evaluación de la interfaz humano-computador permite la comparación de distintas alternativas de diseño de sistemas y, en último lugar, determinar qué sistema es mejor. Por tanto, la evaluación debe proporcionar medidas cuantitativas para determinar la usabilidad de una interfaz y comparar dos interfaces distintas de forma objetiva.
Aunque los usuarios de computadores están habituados al uso de Sistemas de Notificación, su evaluación está limitada a probar el valor intrínseco de paradigmas aislados de diseño basados en una única implementación. La evaluación de usabilidad de los Sistemas de Notificación está muy poco desarrollada en las interfaces de propósito general y, particularmente, en las interfaces de los sistemas críticos. Las únicas experiencias disponibles están basadas en datos empíricos reducidos en entornos simulados y en el seguimiento de ciertas guías de diseño probadas por la experiencia. Están guías proceden de aplicaciones militares y, aunque pueden ser cuestionadas desde el punto de vista de la ciencias cognitivas, están ampliamente aceptadas en el diseño de interfaces para Sistemas de Notificación críticos.
Los diseñadores de estos sistemas pueden, razonablemente, rechazarlas basándose en su experiencia y en su juicio. Si no existe una evaluación cuantitativa basada en modelos cognitivos entonces el evaluador no puede ser capaz de determinar científicamente los errores en la interfaz. De hecho, puede llegarse a la situación sin salida en la que se enfrenten el juicio y el conocimiento del diseñador con los del evaluador.
Esta situación puede resolverse con un marco de evaluación que proporcione datos cuantitativos que muestren si una alternativa de diseño es mejor que otra, basado en principios objetivos.
El diseño y la evaluación de las interfaces humano-computador han evolucionado mucho más rápidamente que las teorías basadas en la psicología y la ciencia cognitiva y aún existen un gran número de problemas por resolver. Así, el diseño de interfaces ha evolucionado desde los editores de líneas y aplicaciones a medida con interfaces muy reducidas basadas en crípticos comandos de línea hasta las interfaces WIMP (Windows-Icons-Menus-Pointers), los sistemas de recuperación de la información, los agentes inteligentes, los servidores CSCW (Cooperative and Shared Computer Works), los entornos virtuales, la realidad aumentada, los bits tangibles, la computación ubicua, la web y los múltiples tipos de interfaces capaces de hablar y mostrar emociones. Mientras tanto, la evaluación de la usabilidad ha evolucionado basada principalmente en heurísticas y en la evaluación empírica [Nielsen 1993; Dix et al. 2.003]. La evaluación basada en la teoría ha sido considerada de un valor muy limitado [Landauer 1987] debido a su reducido ámbito de aplicación, aplicable únicamente a características propias de una interfaz y con una capacidad muy reducida para ser aplicada en otros diseños del mundo real, alejado de los laboratorios. Aunque se están realizando grandes esfuerzos para desarrollar teorías que cubran un amplio rango de tópicos, la mayor parte de ellas son solamente ajustes para explicar aplicaciones del día a día [Rudisill et al. 1996].
Actualmente la medida de prestaciones no se realiza frente a requisitos específicos previos al diseño. Es notable la ausencia de este tipo de requisitos en el ciclo de vida para el desarrollo de una interfaz humano-computador. En estos casos, se puede introducir un conjunto de parámetros críticos que permita definir las unidades de medida [Newman et al. 2000]. Estos parámetros son críticos porque el éxito o el fracaso del diseño dependen de forma crítica del cumplimiento de los objetivos fijados para estos parámetros.
Los parámetros críticos permiten formalizar un espacio de diseño en el que se re-usa el conocimiento [Cheward et al. 2004], expresando los problemas asociados a una interfaz con un lenguaje consistente y que permite a los expertos expresar juicios a través de una ‘evaluación medida’ [Caroll et al. 1992].
El principal objetivo de esta tesis es desarrollar un marco de evaluación predictiva para Sistemas de Notificación en Aviónica (SNA) basado en las teorías cognitivas, que pueda proporcionar datos cualitativos y cuantitativos que permita a los diseñadores y evaluadores de interfaces averiguar si la interfaz sea usable antes de construirlo. De este modo, el diseñador de interfaces puede ser capaz de introducir la evaluación como patrones de diseño de la interfaz humano-computador sin requerir recursos adicionales.
En esta tesis, la aproximación a la evaluación de la interfaz humano-computador de los Sistemas de Notificación para sistemas críticos en aviónica se propone en varios pasos. En primer lugar se establecen un conjunto de parámetros críticos basado en los modelos psicológicos, para después, realizar una evaluación predictiva de la usabilidad basada en la cuantificación de estos parámetros y, finalmente, aplicar de forma incremental las guías de diseño basándose en una estrategia de escenarios. Esta aproximación permitirá una evaluación sistemática garantizando, por un lado, centrarse en el diseño en términos de objetivos cuantificables, independientemente del evaluador y del usuario y, por otro lado, mejorar de forma continua y agregativa los diseños específicos aplicables a un problema determinado.
En esta tesis, la principal contribución es la formalización matemática de cada uno de los parámetros críticos seleccionados para llevar a cabo la evaluación de usabilidad de las interfaces de Sistemas de Notificación integrados en sistemas críticos. El objetivo de esta tesis es conseguir una función de evaluación Ф, tal que aplicada a la interfaz del sistema de notificación permita obtener datos numéricos en el dominio de los números reales.
Formalmente, el objetivo es obtener
ℜΦ∃∀a)(/IIφ
Donde I es la interfaz del sistema de notificación y Ф es la función de evaluación de la usabilidad que produce como resultado un número real.
Para evaluar la validez de este marco de evaluación es aplicado a la interfaz de la planta de potencia de dos aviones modernos y a la interfaz de control de la planta de potencia de un avión de transporte militar.
La aplicación de este marco de evaluación está basada en la ejecución de tests automáticos implementados en Ada95 aplicados a una caracterización XML de las interfaces.
____________________________________________
It’s unheard-of to find any system of any kind, whether critical or not, based on computer with no human interaction. Even though it will be a system related with the communication between two computers it is mostly used to find an interface allowing to solve any problem related with the link between computers [Mulligan et al. 1991].. The user of a computer interface, as any human being, needs information, which is considered one of the few constants in our every day world, whatever his culture or activity. This need plays together with other tasks the user has to perform in such a way that there are at least two systems competing for the most appraised resource at any human-computer interface: the user attention. Different usage situations, expectations, and error consequences govern the growing breed of applications and devices being introduced to support multitasking information demands. Referred to as notification systems, these interfaces are generally desired as a means to access valued information in an efficient and effective manner without introducing unwanted interruption to a primary task. They can be found in many implementation forms and on a variety of platforms. Perhaps classic desktop systems are the most readily identifiable—instant messengers, status programs, and news and stock tickers. Other familiar examples such as Weiser’s dangling string representation of network, in-vehicle information systems, ambient media, multi-monitor displays hint at the potential range of systems and modern aircraft cockpit. While use of these systems and the range of solutions have skyrocketed, our ability to scientifically recognize, pattern, and improve success within the HCI community has not kept pace. There are surprisingly few efforts in the literature that effectively evaluate usability of the information and interaction design for notification systems. On the other hand, a system will be considered as critic when a sole failure due to the system, the user, or the way the user executes the tasks that the system is designed for could produce serious damage to the system itself, in its surrounds or even tragic damage to the user. Examples of this kind of system can be found on many aircrafts corresponding to critical systems handled by very specialized users. Generically usability evaluation includes a spread set of methods and techniques used by evaluators for examining, qualitatively or quantitatively, several aspects related with usability. As support to the design, evaluation will allow firstly verification of design decisions and secondly to choose among possible alternatives. Although evaluation is recognized as very valuable it is rarely introduced in the design cycle of a product and it is deemed as very hard in terms of time and resources and what is worse is that it does not add any value to the product. Human-computer interface evaluation permits the comparison of different systems and ultimately to determine which system is better. Therefore evaluation shall provide direct measurement units to determine the interface usability and to compare two interfaces objectively. Even though computer users are extensively used for notification systems their evaluation is limited to probe the intrinsic value of some isolated paradigms of design based in a single implementation. The evaluation of notification systems is poorly developed on interfaces for systems in general and extremely rare in critical systems. The only available experiences are based on empirical data on simulated environments and the follow-up of some guidebooks probed by the experience. Those guidebooks come from military applications and even they can be questioned from the cognitive science point of view. These guidebooks are mostly accepted on interface user design for notification critical systems. The designer of these interfaces can justifiably reject them based on his experience and judgement. If there is no quantitative evaluation based on cognitive models then the evaluator will never be able to scientifically determine the errors in the interface. A situation can be reached with no way out, with the designer and the evaluator set against each other with conflicting needs. This situation can be resolved with an evaluation framework, which provides quantitative data showing whether one design is better than the other based on objective principles. Design and evaluation of human-computer interfaces have evolved much faster than theories based on psychology and cognitive science and still there are a number of issues that remain unresolved. The design has evolved since line editors and customized application with interfaces reduced to very cryptic line commands up to interfaces WIMP (Windows-Icons-Menus-Pointers), information recovery systems, smart agents, CSCW (Cooperative and Shared Computer Works) servers, virtual environments, augmented reality, tangible bits, ubiquitous computation, the web and many types of interfaces able to talk and show emotions. Usability evaluation has evolved based only on heuristics and empirical evaluation [Nielsen 1993; Dix et al. 2.003]. Evaluation based on theory has been considered as being of very limited value [Landauer 1987] due to the fact that it has a very limited scope and application areas, applicable only to local features of an interface and with a very reduced capacity to be re-used on other designs in the real world, far away from laboratories. Although a big effort is paid for developing theories covering a wide set of topics most of them are only adjustments so as to explain every day applications. Currently, the performance measurements are not done against specific requirements prior to design. It is notable that there is an absence of this kind of requirements in a human-computer interface development life cycle. In case of absence of performance requirements a set of critical parameters could be introduced in order to define measurement units [Newman et al. 2000]. These parameters are so called critical because the success or failure of a design will depend critically whether the fixed objectives are fulfilled or not. Critical parameters allow formalising a design space [Cheward et al. 2004] which allocates acknowledge re-use, expressing interface problems with a consistent language and let experts to express judgements through a mediated evaluation [Caroll et al. 1992]. The main objective of this thesis is to develop a framework of predictive evaluation for notification system on avionics (NSA) (from aviation electronics) based on cognitive theories which can bring both quantitative and qualitative data which allow interface designers and evaluators to guess whether the system will be usable before it is built. The interface designer will be able to include evaluation on the Human-Computer Interface pattern design without adding any additional resources. In this thesis the approach to usability evaluation for the design of Human-Computer Interface of notification systems for critical design in avionics is proposed in several steps. Firstly to establish the set of critical parameters based on psychological models, then to carry out a predictive usability evaluation based on the quantification of these parameters and finally to apply in an incremental way the design guidelines based on a scenario strategy. This approach will allow a systematic evaluation, achieving on one side to focalize the design in term of quantifiable objectives independently of the evaluator and the user and, on other side improving in a continuous and aggregative way the different designs applicable on a specific problem. In this thesis, the main contribution is a mathematical formalization of each critical parameter for carrying out quantitative usability evaluation on interfaces for notification systems integrated on critical system. The objective is to reach an evaluation function Ф in such a way that applied to the interface of a notification system I it will permit to get quantitative data on the real numbers domain. Formally the objective is to obtain ℜΦ∃∀a)(/IIφ Where I is the interface of the notification system and Ф is the usability evaluation function which produce a real number. To evaluate the validity of this framework it will be applied to two power plant control interfaces of modern aircrafts and to a power plant control interface to be implemented in a military transport aircraft. The application of this framework is based on the execution of automatic test implemented on Ada95 applied to a XML characterization for the interfaces.
It’s unheard-of to find any system of any kind, whether critical or not, based on computer with no human interaction. Even though it will be a system related with the communication between two computers it is mostly used to find an interface allowing to solve any problem related with the link between computers [Mulligan et al. 1991].. The user of a computer interface, as any human being, needs information, which is considered one of the few constants in our every day world, whatever his culture or activity. This need plays together with other tasks the user has to perform in such a way that there are at least two systems competing for the most appraised resource at any human-computer interface: the user attention. Different usage situations, expectations, and error consequences govern the growing breed of applications and devices being introduced to support multitasking information demands. Referred to as notification systems, these interfaces are generally desired as a means to access valued information in an efficient and effective manner without introducing unwanted interruption to a primary task. They can be found in many implementation forms and on a variety of platforms. Perhaps classic desktop systems are the most readily identifiable—instant messengers, status programs, and news and stock tickers. Other familiar examples such as Weiser’s dangling string representation of network, in-vehicle information systems, ambient media, multi-monitor displays hint at the potential range of systems and modern aircraft cockpit. While use of these systems and the range of solutions have skyrocketed, our ability to scientifically recognize, pattern, and improve success within the HCI community has not kept pace. There are surprisingly few efforts in the literature that effectively evaluate usability of the information and interaction design for notification systems. On the other hand, a system will be considered as critic when a sole failure due to the system, the user, or the way the user executes the tasks that the system is designed for could produce serious damage to the system itself, in its surrounds or even tragic damage to the user. Examples of this kind of system can be found on many aircrafts corresponding to critical systems handled by very specialized users. Generically usability evaluation includes a spread set of methods and techniques used by evaluators for examining, qualitatively or quantitatively, several aspects related with usability. As support to the design, evaluation will allow firstly verification of design decisions and secondly to choose among possible alternatives. Although evaluation is recognized as very valuable it is rarely introduced in the design cycle of a product and it is deemed as very hard in terms of time and resources and what is worse is that it does not add any value to the product. Human-computer interface evaluation permits the comparison of different systems and ultimately to determine which system is better. Therefore evaluation shall provide direct measurement units to determine the interface usability and to compare two interfaces objectively. Even though computer users are extensively used for notification systems their evaluation is limited to probe the intrinsic value of some isolated paradigms of design based in a single implementation. The evaluation of notification systems is poorly developed on interfaces for systems in general and extremely rare in critical systems. The only available experiences are based on empirical data on simulated environments and the follow-up of some guidebooks probed by the experience. Those guidebooks come from military applications and even they can be questioned from the cognitive science point of view. These guidebooks are mostly accepted on interface user design for notification critical systems. The designer of these interfaces can justifiably reject them based on his experience and judgement. If there is no quantitative evaluation based on cognitive models then the evaluator will never be able to scientifically determine the errors in the interface. A situation can be reached with no way out, with the designer and the evaluator set against each other with conflicting needs. This situation can be resolved with an evaluation framework, which provides quantitative data showing whether one design is better than the other based on objective principles. Design and evaluation of human-computer interfaces have evolved much faster than theories based on psychology and cognitive science and still there are a number of issues that remain unresolved. The design has evolved since line editors and customized application with interfaces reduced to very cryptic line commands up to interfaces WIMP (Windows-Icons-Menus-Pointers), information recovery systems, smart agents, CSCW (Cooperative and Shared Computer Works) servers, virtual environments, augmented reality, tangible bits, ubiquitous computation, the web and many types of interfaces able to talk and show emotions. Usability evaluation has evolved based only on heuristics and empirical evaluation [Nielsen 1993; Dix et al. 2.003]. Evaluation based on theory has been considered as being of very limited value [Landauer 1987] due to the fact that it has a very limited scope and application areas, applicable only to local features of an interface and with a very reduced capacity to be re-used on other designs in the real world, far away from laboratories. Although a big effort is paid for developing theories covering a wide set of topics most of them are only adjustments so as to explain every day applications. Currently, the performance measurements are not done against specific requirements prior to design. It is notable that there is an absence of this kind of requirements in a human-computer interface development life cycle. In case of absence of performance requirements a set of critical parameters could be introduced in order to define measurement units [Newman et al. 2000]. These parameters are so called critical because the success or failure of a design will depend critically whether the fixed objectives are fulfilled or not. Critical parameters allow formalising a design space [Cheward et al. 2004] which allocates acknowledge re-use, expressing interface problems with a consistent language and let experts to express judgements through a mediated evaluation [Caroll et al. 1992]. The main objective of this thesis is to develop a framework of predictive evaluation for notification system on avionics (NSA) (from aviation electronics) based on cognitive theories which can bring both quantitative and qualitative data which allow interface designers and evaluators to guess whether the system will be usable before it is built. The interface designer will be able to include evaluation on the Human-Computer Interface pattern design without adding any additional resources. In this thesis the approach to usability evaluation for the design of Human-Computer Interface of notification systems for critical design in avionics is proposed in several steps. Firstly to establish the set of critical parameters based on psychological models, then to carry out a predictive usability evaluation based on the quantification of these parameters and finally to apply in an incremental way the design guidelines based on a scenario strategy. This approach will allow a systematic evaluation, achieving on one side to focalize the design in term of quantifiable objectives independently of the evaluator and the user and, on other side improving in a continuous and aggregative way the different designs applicable on a specific problem. In this thesis, the main contribution is a mathematical formalization of each critical parameter for carrying out quantitative usability evaluation on interfaces for notification systems integrated on critical system. The objective is to reach an evaluation function Ф in such a way that applied to the interface of a notification system I it will permit to get quantitative data on the real numbers domain. Formally the objective is to obtain ℜΦ∃∀a)(/IIφ Where I is the interface of the notification system and Ф is the usability evaluation function which produce a real number. To evaluate the validity of this framework it will be applied to two power plant control interfaces of modern aircrafts and to a power plant control interface to be implemented in a military transport aircraft. The application of this framework is based on the execution of automatic test implemented on Ada95 applied to a XML characterization for the interfaces.
Description
Keywords
Interfaces de usuario, Aeronáutica, Sistemas de notificación, Usabilidad