Publication: A Data-Aware Scheduling Strategy for DMCF workflows over Hercules
Loading...
Identifiers
Publication date
2016-10-06
Defense date
Advisors
Tutors
Journal Title
Journal ISSN
Volume Title
Publisher
Impact
Export
Abstract
As data-intensive scientific prevalence arises, there is a necessity of simplifying the development, deployment, and execution of complex data analysis applications. The Data Mining Cloud Framework is a service-oriented system for allowing users to design and execute data analysis applications, defined as workflows, on cloud platforms, relying on cloud-provided storage services for I/O operations. Hercules is an in-memory I/O solution that can be deployed as an alternative to cloud storage services, providing additional performance and flexibility features. This work extends the DMCF-Hercules cooperation by applying novel data placement and task scheduling techniques for exposing and exploiting data locality in data-intensive workflows.
Description
Proceedings of: Third International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2016). Sofia (Bulgaria), October, 6-7, 2016.
Keywords
DMCF, Hercules, Workflows, In-memory storage, Data cache, Microsoft Azure, Data locality
Bibliographic citation
Carretero Pérez, Jesús; et.al. (eds.). (2016) Proceedings of the Third International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2016): Sofia, Bulgaria. Universidad Carlos III de Madrid, pp. 37-44.