Publication: Elementos locales en conjuntos de clasificadores diseñados por "Boosting"
Loading...
Identifiers
Publication date
2013-04
Defense date
2013-04-04
Authors
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
Aunque los diseños bloque de Máquinas de Aprendizaje son en general una herramienta poderosa para resolver problemas de inferencia, en situaciones complicadas pueden resultar insuficientes. Para solventar esta limitación cabe considerar la combinacióon de diferentes máquinas (simples) de modo que se forme un conjunto capaz de resolver mejor el problema en cuestión, proporcionando, además, un diseño más sencillo y más fácilmente comprensible. Entre los conjuntos destacan, por sus sencillos principios conceptuales y sus contrastadas buenas prestaciones, los métodos de Boosting, y, especialmente, el algoritmo “Real AdaBoost” (RAB). A pesar de que las características mencionadas han llevado a que el empleo de los métodos de Boosting sea cada vez más frecuente, su diseño está limitado, por un lado, porque los aprendices que componen el conjunto se combinan linealmente, y, por otro lado, al uso de clasificadores base de tipo global o, de ser locales, con limitada capacidad expresiva. Pues bien: el objetivo de esta Tesis Doctoral es mejorar las prestaciones de los métodos de Boosting introduciendo elementos locales con estructuras de núcleos en los aprendices o en su fusión, con el objeto de paliar las citadas limitaciones. Así, en primer lugar, se presenta y aplica un esquema de fusión controlado por puertas similares a las empleadas en las Mezclas de Expertos. Este nuevo modelo de fusión utiliza una puerta entrenable compuesta por un cuerpo con estructuras de núcleos sencillos y un vector de pesos de salida que se ajusta paso a paso minimizando la función de coste del RAB, sin necesidad de realizar modificaciones en el modo de entrenar los aprendices. Los resultados obtenidos indican que estos esquemas permiten frecuentemente alcanzar mejores prestaciones con baja carga computacional en operación, si bien a cambio de un sensible incremento de carga computacional en diseño debido a la necesidad de determinar mediante validación cruzada los valores de los parámetros propios de la puerta. En segundo lugar, se introduce un nuevo procedimiento que permite el uso de Máquinas de Vectores Soporte (SVMs) como aprendices de conjuntos construidos mediante Boosting. Se diseñan estos aprendices aplicando un procedimiento de submuestreo en una SVM con Programación Lineal, lo que posibilita el uso de una matriz núcleo submuestreada por filas, con la consiguiente flexibilidad a la hora de controlar la diversidad y complejidad de este tipo de aprendices. Esta combinación de modificaciones de diseño ha dado como resultado conjuntos con estructuras de núcleos reutilizables por los diferentes aprendices, característica que ha permitido conseguir máquinas compactas de muy buenas prestaciones y con una carga computacional de operación similar a la de una SVM individual. El trabajo se cierra con una revisión de las aportaciones y sugerencias de líneas de investigación abiertas. --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Although the block design of learning machines is, in general, a powerful tool to solve inference problems, it can be insufficient in complex scenarios. In order to solve this limitation, it is necessary to consider combinations of different (simple) machines, that form an ensemble which is not only able to solve the problem at hand but also to provide a simpler and somewhat more understandable design. Among different ensemble building techniques, Boosting methods and, in particular, the “Real AdaBoost” (RAB) algorithm prevail due to their simple conceptual principles and their well-known good performance. Despite the above-mentioned characteristics have caused an increasing use of Boosting methods, their designs are limited. A first limitations comes from the linear combination of the ensemble learners, and, on the other hand, by the use of global base classifiers or, when including traditional local learners, by their limited expressive power. The objective of this Ph.D. Thesis is just to improve the performance of Boosting methods by introducing local elements with a kernel-based structure either in the learners or in their fusion, with the purpose of reducing the effect of the mentioned limitations. In this way, firstly, a method based on including fusion gates similar to those used in Mixtures of Experts, is presented and applied. This new fusion model employs a trainable gate consisting of a set of simple kernel structures and an output weight vector which is step-by-step adjusted so that the RAB cost function is minimized, without the need of modifying the learner training. Obtained results evidence that these schemes frequently achieve performance improvements with a low operation computational load, but with a significant computational burden increment in the design process due to the need of determining the gate parameters by a cross validation procedure. Secondly, a new procedure to allow the use of Support Vector Machines (SVM) as base learners of Boosting ensembles is presented. These learners are designed by applying a sub-sampling procedure in a Linear Programming SVM, making possible the use of a kernel matrix subsampled by rows and, therefore, providing flexibility to control the diversity and complexity of this kind of learners. These design modifications have provided ensembles with kernel structures which are reusable by the different learners. This relevant characteristic allows to achieve compact machines with accurate performances and a computational load similar to that of a single SVM. This document concludes with a review of the main contributions and some suggestions about new research lines arising from this work.
Although the block design of learning machines is, in general, a powerful tool to solve inference problems, it can be insufficient in complex scenarios. In order to solve this limitation, it is necessary to consider combinations of different (simple) machines, that form an ensemble which is not only able to solve the problem at hand but also to provide a simpler and somewhat more understandable design. Among different ensemble building techniques, Boosting methods and, in particular, the “Real AdaBoost” (RAB) algorithm prevail due to their simple conceptual principles and their well-known good performance. Despite the above-mentioned characteristics have caused an increasing use of Boosting methods, their designs are limited. A first limitations comes from the linear combination of the ensemble learners, and, on the other hand, by the use of global base classifiers or, when including traditional local learners, by their limited expressive power. The objective of this Ph.D. Thesis is just to improve the performance of Boosting methods by introducing local elements with a kernel-based structure either in the learners or in their fusion, with the purpose of reducing the effect of the mentioned limitations. In this way, firstly, a method based on including fusion gates similar to those used in Mixtures of Experts, is presented and applied. This new fusion model employs a trainable gate consisting of a set of simple kernel structures and an output weight vector which is step-by-step adjusted so that the RAB cost function is minimized, without the need of modifying the learner training. Obtained results evidence that these schemes frequently achieve performance improvements with a low operation computational load, but with a significant computational burden increment in the design process due to the need of determining the gate parameters by a cross validation procedure. Secondly, a new procedure to allow the use of Support Vector Machines (SVM) as base learners of Boosting ensembles is presented. These learners are designed by applying a sub-sampling procedure in a Linear Programming SVM, making possible the use of a kernel matrix subsampled by rows and, therefore, providing flexibility to control the diversity and complexity of this kind of learners. These design modifications have provided ensembles with kernel structures which are reusable by the different learners. This relevant characteristic allows to achieve compact machines with accurate performances and a computational load similar to that of a single SVM. This document concludes with a review of the main contributions and some suggestions about new research lines arising from this work.
Description
Keywords
Aprendizaje máquina, Inteligencia artificial, Boosting, Algoritmos, Real AdaBoost, RAB, Máquinas de vectores soporte