mapreduce framework optimization via performance modeling

ISCAS OpenIR

	mapreduce framework optimization via performance modeling
	Xu Lijie
	2012
会议名称	2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012
会议录名称	Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012
页码	2506-2509
会议日期	May 21, 2012 - May 25, 2012
会议地点	Shanghai, China
收录类别	EI ; ISTP
ISSN	2164-7062
ISBN	9780769546766
部门归属	(1) Institute of Software Chinese Academy of Sciences China; (2) Graduate University Chinese Academy of Sciences China
摘要	MapReduce framework has become the state-of-the-art paradigm for large-scale data processing. In our ongoing work, we attempt to solve the three interrelated problems: how to build an accurate MapReduce performance model, how to use it to automatically detect and optimize slow-running MapReduce jobs, and how to use it to help scheduler arrange job execution sequence. Currently, we mainly study the job execution time model and its training method. We also present several policies to optimize the job configuration and scheduler. © 2012 IEEE.; MapReduce framework has become the state-of-the-art paradigm for large-scale data processing. In our ongoing work, we attempt to solve the three interrelated problems: how to build an accurate MapReduce performance model, how to use it to automatically detect and optimize slow-running MapReduce jobs, and how to use it to help scheduler arrange job execution sequence. Currently, we mainly study the job execution time model and its training method. We also present several policies to optimize the job configuration and scheduler. © 2012 IEEE.
关键词	Data Processing Distributed Parameter Networks Scheduling
主办者	IEEE Computer Society Technical Committee on Parallel Processing
学科领域	Computer Science
语种	英语
内容类型	会议论文
URI标识	http://ir.iscas.ac.cn/handle/311060/15782
专题	中国科学院软件研究所
推荐引用方式 GB/T 7714	Xu Lijie. mapreduce framework optimization via performance modeling[C],2012:2506-2509.