Exploiting data locality in Swift/T workflows using Hercules

e-Archivo Repository

Show simple item record

dc.contributor.author Rodrigo Duro, Francisco José
dc.contributor.author García Blas, Javier
dc.contributor.author Isaila, Florín Daniel
dc.contributor.author Carretero Pérez, Jesús
dc.contributor.author Wozniak, Justin M.
dc.contributor.author Ross, Rob
dc.contributor.editor Carretero Pérez, Jesús
dc.contributor.editor García Blas, Javier
dc.contributor.editor Barbosa, Jorge
dc.contributor.editor Morla, Ricardo
dc.contributor.other Universidad Carlos III de Madrid. Computer Architecture, Communications and Systems Group (ARCOS)
dc.date.accessioned 2015-10-29T12:18:45Z
dc.date.available 2015-10-29T12:18:45Z
dc.date.issued 2014-11
dc.identifier.bibliographicCitation Carretero Pérez, Jesús; et.al. (eds.). (2014) Proceedings of the First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014): Porto, Portugal. Universidad Carlos III de Madrid, pp. 71-76.
dc.identifier.isbn 978-84-617-2251-8
dc.identifier.uri http://hdl.handle.net/10016/21865
dc.description.abstract The ever-increasing power of supercomputer systems is both driving and enabling the emergence of new problem-solving methods that require the efficient execution of many concurrent and interacting tasks. Swift/T, as a description language and runtime, offers the dynamic creation and execution of workflows, varying in granularity, on high-component-count platforms. Swift/T takes advantage of the Asynchronous Dynamic Load Balancing (ADLB) library to dynamically distribute the tasks among the nodes. These tasks may share data using a parallel file system, an approach that could degrade performance as a result of interference with other applications and poor exploitation of data locality. The objective of this work is to expose and exploit data locality in Swift/T through Hercules, a distributed in-memory store based on Memcached, and to explore tradeoffs between data locality and load balance in distributed workflow executions. In this paper we present our approach to enable locality-based optimizations in Swift/T by guiding ADLB to schedule computation jobs in the nodes containing the required data. We also analyze the interaction between locality and load balance: our initial measurements based on various raw file access patterns show promising results. Moreover, we present future work based on the promising results achieved so far.
dc.description.sponsorship This material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract DE-AC02-06CH11357. Computing resources were provided by the Argonne Leadership Computing Facility. The work presented in this paper was supported by the COST Action IC1305, “Network for Sustainable Ultrascale Computing (NESUS).” The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement number 328582.
dc.format.extent 6
dc.format.mimetype application/pdf
dc.language.iso eng
dc.subject.other Locality
dc.subject.other In-memory storage
dc.subject.other Swift/T
dc.subject.other Workflows
dc.title Exploiting data locality in Swift/T workflows using Hercules
dc.type bookPart
dc.type conferenceObject
dc.subject.eciencia Informática
dc.rights.accessRights openAccess
dc.relation.projectID info:eu-repo/grantAgreement/EC/FP7/328582
dc.type.version publishedVersion
dc.relation.eventdate August 27-28, 2014
dc.relation.eventnumber 1
dc.relation.eventplace Porto, Portugal
dc.relation.eventtitle First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014)
dc.relation.eventtype proceeding
dc.identifier.publicationfirstpage 71
dc.identifier.publicationlastpage 76
dc.identifier.publicationtitle Proceedings of the First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014): Porto, Portugal
dc.identifier.uxxi CC/0000023994
 Find Full text

Files in this item

*Click on file's image for preview. (Embargoed files's preview is not supported)


This item appears in the following Collection(s)

Show simple item record