RT Conference Proceedings
T1 Exploiting data locality in Swift/T workflows using Hercules
A1 Rodrigo Duro, Francisco José
A1 García Blas, Javier
A1 Isaila, Florín Daniel
A1 Carretero Pérez, Jesús
A1 Wozniak, Justin M.
A1 Ross, Rob
A2 Carretero Pérez, Jesús
A2 García Blas, Javier
A2 Barbosa, Jorge
A2 Morla, Ricardo
A2 Universidad Carlos III de Madrid. Computer Architecture, Communications and Systems Group (ARCOS)
AB The ever-increasing power of supercomputer systems is both driving and enabling the emergence of new problem-solving methods that require the efficient execution of many concurrent and interacting tasks. Swift/T, as a description language and runtime, offers the dynamic creation and execution of workflows, varying in granularity, on high-component-count platforms. Swift/T takes advantage of the Asynchronous Dynamic Load Balancing (ADLB) library to dynamically distribute the tasks among the nodes. These tasks may share data using a parallel file system, an approach that could degrade performance as a result of interference with other applications and poor exploitation of data locality. The objective of this work is to expose and exploit data locality in Swift/T through Hercules, a distributed in-memory store based on Memcached, and to explore tradeoffs between data locality and load balance in distributed workflow executions. In this paper we present our approach to enable locality-based optimizations in Swift/T by guiding ADLB to schedule computation jobs in the nodes containing the required data. We also analyze the interaction between locality and load balance: our initial measurements based on various raw file access patterns show promising results. Moreover, we present future work based on the promising results achieved so far.
SN 978-84-617-2251-8
YR 2014
FD 2014-11
LK https://hdl.handle.net/10016/21865
UL https://hdl.handle.net/10016/21865
LA eng
NO This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract DE-AC02-06CH11357. Computing resources were provided by the Argonne Leadership Computing Facility. The work presented in this paper was supported by the COST Action IC1305, “Network for Sustainable UltrascaleComputing (NESUS).” The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement number 328582.
DS e-Archivo
RD 17 jul. 2024