Home
Resource
Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets
PDF
PDF format is widely accepted and good for printing.
Plug-in required
PDF-1(682.91 kb)
Citation & Export
View Usage Statistics
Staff View

Citation & Export
Hide

Simple citation

Gadre, Hrishikesh. Investigating MapReduce framework extensions for efficient processing of geographically scattered datasets. Retrieved from https://doi.org/doi:10.7282/T3CR5SFB

Export

Click here for information about Citation Management Tools at Rutgers.

Statistics
Hide

Description

TitleInvestigating MapReduce framework extensions for efficient processing of geographically scattered datasets
NameGadre, Hrishikesh (author); Parashar, Manish (chair); Marsic, Ivan (internal member); Rodero, Ivan (outside member); Rutgers University; Graduate School - New Brunswick
Date Created2011
Other Date2011-10 (degree)
SubjectElectrical and Computer Engineering, Information storage and retrieval systems, Data processing service centers
Extentix, 80 p. : ill.
DescriptionWe observe two important trends brought about by the evolution of Internet in recent years. Firstly to improve end-to-end application performance in presence of bottlenecks in the wide-area Internet communication, modern day Internet services are designed in a decentralized fashion involving geographically distributed data-centers connected through the Internet. Secondly the pervasive nature of Internet services has resulted into an exponential growth in the size of digital information created, captured or replicated. Organizations are keenly interested in mining this information to uncover trends, statistics and other actionable information which can give them competitive advantage. These two trends necessitate the design of a large-scale data processing system which can operate efficiently in a distributed environment involving multiple datacenters connected through the Internet. In recent years, MapReduce programming model and specifically its open source implementation Hadoop is gaining a lot of traction for performing large-scale data processing in a centralized environment. Our evaluation of different real-world usage scenarios of Hadoop deployments revealed that the organizations with the distributed datasets are required to copy the entire dataset to a centralized location so that it can be efficiently processed by the Hadoop MapReduce framework. As the Internet evolves growth in the size of distributed datasets would outpace the improvements in the network bandwidth available in the Internet. At that point the approach of copying the entire dataset to a single location using Internet would become infeasible. In this thesis, we have investigated the possibility of extending the MapReduce and specifically Hadoop framework to operate in a distributed environment involving multiple datacenters connected through the Internet. We also have proposed policies to improve the performance of Hadoop MapReduce framework in a distributed environment. We have observed that our policies improve the performance of Hadoop framework substantially.
NoteM.S.
NoteIncludes bibliographical references
Noteby Hrishikesh Gadre
Genretheses, ETD graduate
Persistent URLhttps://doi.org/doi:10.7282/T3CR5SFB
Languageeng
CollectionGraduate School - New Brunswick Electronic Theses and Dissertations
Organization NameRutgers, The State University of New Jersey
RightsThe author owns the copyright to this work.
Version 8.5.5
Rutgers University Libraries - Copyright ©2024