Publication:
Exploiting data locality in Swift/T workflows using Hercules

Simple item page
dc.affiliation.dptoUC3M. Departamento de Informáticaes
dc.affiliation.grupoinvUC3M. Grupo de Investigación: Arquitectura de Computadores, Comunicaciones y Sistemases
dc.contributor.authorRodrigo Duro, Francisco José
dc.contributor.authorGarcía Blas, Javier
dc.contributor.authorIsaila, Florín Daniel
dc.contributor.authorCarretero Pérez, Jesús
dc.contributor.authorWozniak, Justin M.
dc.contributor.authorRoss, Rob
dc.contributor.editorCarretero Pérez, Jesús
dc.contributor.editorGarcía Blas, Javier
dc.contributor.editorBarbosa, Jorge
dc.contributor.editorMorla, Ricardo
dc.contributor.otherUniversidad Carlos III de Madrid. Computer Architecture, Communications and Systems Group (ARCOS)
dc.date.accessioned2015-10-29T12:18:45Z
dc.date.available2015-10-29T12:18:45Z
dc.date.issued2014-11
dc.description.abstractThe ever-increasing power of supercomputer systems is both driving and enabling the emergence of new problem-solving methods that require the efficient execution of many concurrent and interacting tasks. Swift/T, as a description language and runtime, offers the dynamic creation and execution of workflows, varying in granularity, on high-component-count platforms. Swift/T takes advantage of the Asynchronous Dynamic Load Balancing (ADLB) library to dynamically distribute the tasks among the nodes. These tasks may share data using a parallel file system, an approach that could degrade performance as a result of interference with other applications and poor exploitation of data locality. The objective of this work is to expose and exploit data locality in Swift/T through Hercules, a distributed in-memory store based on Memcached, and to explore tradeoffs between data locality and load balance in distributed workflow executions. In this paper we present our approach to enable locality-based optimizations in Swift/T by guiding ADLB to schedule computation jobs in the nodes containing the required data. We also analyze the interaction between locality and load balance: our initial measurements based on various raw file access patterns show promising results. Moreover, we present future work based on the promising results achieved so far.en
dc.description.sponsorshipThis material is based upon work supported by the U.S. Department of Energy, Office of Science, under contract DE-AC02-06CH11357. Computing resources were provided by the Argonne Leadership Computing Facility. The work presented in this paper was supported by the COST Action IC1305, “Network for Sustainable Ultrascale Computing (NESUS).” The research leading to these results has received funding from the European Union Seventh Framework Programme (FP7/2007-2013) under grant agreement number 328582.en
dc.format.extent6
dc.format.mimetypeapplication/pdf
dc.identifier.bibliographicCitationCarretero Pérez, Jesús; et.al. (eds.). (2014) Proceedings of the First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014): Porto, Portugal. Universidad Carlos III de Madrid, pp. 71-76.
dc.identifier.isbn978-84-617-2251-8
dc.identifier.publicationfirstpage71
dc.identifier.publicationlastpage76
dc.identifier.publicationtitleProceedings of the First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014): Porto, Portugalen
dc.identifier.urihttps://hdl.handle.net/10016/21865
dc.identifier.uxxiCC/0000023994
dc.language.isoeng
dc.relation.eventdateAugust 27-28, 2014en
dc.relation.eventnumber1
dc.relation.eventplacePorto, Portugalen
dc.relation.eventtitleFirst International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014)en
dc.relation.projectIDinfo:eu-repo/grantAgreement/EC/FP7/328582
dc.rights.accessRightsopen access
dc.subject.ecienciaInformáticaes
dc.subject.otherLocalityen
dc.subject.otherIn-memory storageen
dc.subject.otherSwift/Ten
dc.subject.otherWorkflowsen
dc.titleExploiting data locality in Swift/T workflows using Herculesen
dc.typeconference paper*
dc.type.hasVersionVoR*
dspace.entity.typePublication
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
exploiting_NESUS_2014.pdf
Size:
589.63 KB
Format:
Adobe Portable Document Format
Download
Collections
First International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2014)
OpenAIRE: Open Access Infrastructure for Research in Europe