Publication: Desarrollo de una herramienta software para la extracción de datos sobre el rendimiento de la red eléctrica
Loading...
Identifiers
Publication date
2015-10-06
Defense date
2015-10-06
Authors
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
El presente trabajo fin de grado se enmarca dentro del contexto de las herramientas software para la extracción de datos o “data scraping”, lo que en castellano podría traducirse como “rascado” o “raspado” de información. Esta extracción de información a través de herramientas software o “data scrapers” se realiza mediante la aplicación de diversas técnicas en las que un ordenador extrae información legible por el ser humano a partir de un conjunto de datos en bruto. Estos data scrapers se pueden dividir en dos tipos en función de la fuente de la información utilizada, los “screen scrapers”, los cuáles extraen información de una fuente visual como puede ser un terminal de ordenador y los “web scrapers”, los cuáles utilizan los lenguajes de marcado como HTML para obtener información. Es en este segundo tipo de “data scraper” en el cual se focaliza este trabajo. Esto se debe a la necesidad dada por el departamento de Ingeniería Eléctrica de la Universidad Carlos III de Madrid de extraer información sobre el consumo eléctrico desde la página web de Red Eléctrica de España para su posterior análisis.
El consumo eléctrico es una constante en nuestra vida diaria, tanto para nuestro uso personal como indirectamente para la industria o el sector servicios. Dicho consumo, al producirse y contabilizarse, genera datos que son acumulados y pueden ser estudiados con distintos propósitos. El gran obstáculo de este proceso reside en la dificultad de la extracción de estos datos de forma masiva y estructurada, mediante los medios proporcionados por Red Eléctrica de España. A fin de solventar esta problemática, el presente trabajo fin de grado provee una herramienta software que logra la extracción y agregación eficiente de grandes volúmenes de datos sobre el rendimiento de la red eléctrica y su exportación a archivos pretratados para su posterior uso en herramientas de cálculo, así como su visualización.
La evaluación del cubrimiento de los objetivos definidos en el trabajo se ha realizado a través de la realización de distintos casos de prueba. Adicionalmente, se ha realizado un cuestionario de usabilidad a un grupo cerrado de diez usuarios a fin de determinar la facilidad de uso y aprendizaje de la herramienta.
This undergraduate final project is framed within the context of “data scraping”, which is a technique in which a computer extracts data from human-readable output coming from another program. Accordingly, data scrapers are software tools designed to collect data from different sources in an automated way. They can be divided into two types depending on the source, "screen scrapers", which are associated to the collection of visual data from a source such as a computer display, and "web scrapers", which use text-based mark-up languages such as HTML to extract data. This project is focused on the latter given the need of the Electrical Engineering Department of Carlos III University of extracting power consumption-related data from the Red Eléctrica de España public website in order to analyze it. Power consumption is a constant in our daily lives, both for our personal use and, indirectly, for the industry and services sector. This consumption is recorded while is being produced and counted, which can be used afterwards for different purposes. The major obstacle to this process is the difficulty of extracting this power consumption-related data in bulk and in a structured way from the Red Eléctrica de España public website. To solve this problem, this final project provides a software tool that not only manages the aggregation, extraction, and export of large volumes of data for later use in calculation tools but also the visualization of such data. The evaluation of the objectives defined in this project has been carried out through conduction of different test cases. In addition to that, a usability testing with ten users has been also conducted to determine the ease of use and learning of the proposed software tool.
This undergraduate final project is framed within the context of “data scraping”, which is a technique in which a computer extracts data from human-readable output coming from another program. Accordingly, data scrapers are software tools designed to collect data from different sources in an automated way. They can be divided into two types depending on the source, "screen scrapers", which are associated to the collection of visual data from a source such as a computer display, and "web scrapers", which use text-based mark-up languages such as HTML to extract data. This project is focused on the latter given the need of the Electrical Engineering Department of Carlos III University of extracting power consumption-related data from the Red Eléctrica de España public website in order to analyze it. Power consumption is a constant in our daily lives, both for our personal use and, indirectly, for the industry and services sector. This consumption is recorded while is being produced and counted, which can be used afterwards for different purposes. The major obstacle to this process is the difficulty of extracting this power consumption-related data in bulk and in a structured way from the Red Eléctrica de España public website. To solve this problem, this final project provides a software tool that not only manages the aggregation, extraction, and export of large volumes of data for later use in calculation tools but also the visualization of such data. The evaluation of the objectives defined in this project has been carried out through conduction of different test cases. In addition to that, a usability testing with ten users has been also conducted to determine the ease of use and learning of the proposed software tool.
Description
Keywords
Herramientas software, Extracción de datos, Data scraping, Screen scrapers, Red Eléctrica de España, Energía eléctrica