Log File Analysis in Cloud with Apache Hadoop and Apache Spark

e-Archivo Repository

Show simple item record

dc.contributor.author Mavridis, Ilias
dc.contributor.author Karatza, Eleni
dc.contributor.editor Carretero Pérez, Jesús
dc.contributor.editor García Blas, Javier
dc.contributor.editor Wyrzykowski, Roman
dc.contributor.editor Jeannot, Emmanuel
dc.contributor.other Universidad Carlos III de Madrid. Computer Architecture, Communications and Systems Group (ARCOS)
dc.date.accessioned 2015-11-12T11:50:22Z
dc.date.available 2015-11-12T11:50:22Z
dc.date.issued 2015-10
dc.identifier.bibliographicCitation Carretero Pérez, Jesús; et.al. (eds.). (2015) Proceedings of the Second International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015): Krakow, Poland. Universidad Carlos III de Madrid, pp. 51-62.
dc.identifier.isbn 978-84-608-2581-4
dc.identifier.uri http://hdl.handle.net/10016/21995
dc.description Proceedings of: Second International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015). Krakow (Poland), September 10-11, 2015.
dc.description.abstract Log files are a very important set of data that can lead to useful information through proper analysis. Due to the high production rate and the number of devices and software that generate logs, the use of cloud services for log analysis is almost necessary. This paper reviews the cloud computational framework ApacheTM Hadoop R, highlights the differences and similarities between Hadoop MapReduce and Apache SparkTM and evaluates the performance of them. Log file analysis applications were developed in both frameworks and performed SQL-type queries in real Apache Web Server log files. Various measurements were taken for each application and query with different parameters in order to extract safe conclusions about the performance of the two frameworks.
dc.description.sponsorship The authors would like to thank Okeanos the GRNET’s cloud service for the valuable resources.
dc.format.extent 12
dc.format.mimetype application/pdf
dc.language.iso eng
dc.subject.other Log analysis
dc.subject.other Cloud
dc.subject.other Apache hadoop
dc.subject.other Apache spark
dc.subject.other Performance evaluation
dc.title Log File Analysis in Cloud with Apache Hadoop and Apache Spark
dc.type bookPart
dc.type conferenceObject
dc.subject.eciencia Informática
dc.rights.accessRights openAccess
dc.type.version publishedVersion
dc.relation.eventdate September 10-11, 2015
dc.relation.eventnumber 2
dc.relation.eventplace Krakow, Poland
dc.relation.eventtitle International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015)
dc.relation.eventtype proceeding
dc.identifier.publicationfirstpage 51
dc.identifier.publicationlastpage 62
dc.identifier.publicationtitle Proceedings of the Second International Workshop on Sustainable Ultrascale Computing Systems (NESUS 2015): Krakow, Poland
 Find Full text

Files in this item

*Click on file's image for preview. (Embargoed files's preview is not supported)


This item appears in the following Collection(s)

Show simple item record