RT Conference Proceedings
T1 Evaluating data caching techniques in DMCF workflows using Hercules
A1 Rodrigo Duro, Francisco José
A1 Marozzo, Fabrizio
A1 García Blas, Javier
A1 Carretero Pérez, Jesús
A1 Talia, Domenico
A1 Trunfio, Paolo
AB The Data Mining Cloud Framework (DMCF) is an environment for designing and executing data analysis workflows in cloud platforms. Currently, DMCF relies on the default storage of the public cloud provider for any I/O related operation. This implies that the I/O performance of DMCF is limited by the performance of the default storage. In this work we propose the usage of the Hercules system within DMCF as an ad-hoc storage system for temporary data produced inside workflow-based applications. Hercules is a distributed in-memory storage system highly scalable and easy to deploy. The proposed solution takes advantage of the scalability capabilities of Hercules to avoid the bandwidth limits of the default storage. Early experimental results are presented in this paper, they show promising performance, particularly for write operations, compared to the performance obtained using the default storage services.
SN 978-84-608-2581-4
YR 2015
FD 2015-10
LK https://hdl.handle.net/10016/22027
UL https://hdl.handle.net/10016/22027
LA eng
NO This work is partially supported by EU under the COST Program Action IC1305: Network for Sustainable Ultrascale Computing (NESUS). This work is partially supported by the grant TIN2013-41350-P, Scalable Data Management Techniques for High-End Computing Systems from the Spanish Ministry of Economy and Competitiveness.
DS e-Archivo
RD 3 may. 2024