Institutional Repository
| mapreduce framework optimization via performance modeling | |
| Xu Lijie | |
| 2012 | |
| Conference Name | 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012 |
| Source | Proceedings of the 2012 IEEE 26th International Parallel and Distributed Processing Symposium Workshops, IPDPSW 2012 |
| Pages | 2506-2509 |
| Conference Date | May 21, 2012 - May 25, 2012 |
| Conference Place | Shanghai, China |
| Indexed Type | EI ; ISTP |
| ISSN | 2164-7062 |
| ISBN | 9780769546766 |
| Department | (1) Institute of Software Chinese Academy of Sciences China; (2) Graduate University Chinese Academy of Sciences China |
| English Abstract | MapReduce framework has become the state-of-the-art paradigm for large-scale data processing. In our ongoing work, we attempt to solve the three interrelated problems: how to build an accurate MapReduce performance model, how to use it to automatically detect and optimize slow-running MapReduce jobs, and how to use it to help scheduler arrange job execution sequence. Currently, we mainly study the job execution time model and its training method. We also present several policies to optimize the job configuration and scheduler. © 2012 IEEE.; MapReduce framework has become the state-of-the-art paradigm for large-scale data processing. In our ongoing work, we attempt to solve the three interrelated problems: how to build an accurate MapReduce performance model, how to use it to automatically detect and optimize slow-running MapReduce jobs, and how to use it to help scheduler arrange job execution sequence. Currently, we mainly study the job execution time model and its training method. We also present several policies to optimize the job configuration and scheduler. © 2012 IEEE. |
| Keyword | Data Processing Distributed Parameter Networks Scheduling |
| Sponsorship | IEEE Computer Society Technical Committee on Parallel Processing |
| Subject | Computer Science |
| Language | 英语 |
| Content Type | 会议论文 |
| URI | http://ir.iscas.ac.cn/handle/311060/15782 |
| Collection | 中国科学院软件研究所 |
| Recommended Citation GB/T 7714 | Xu Lijie. mapreduce framework optimization via performance modeling[C],2012:2506-2509. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment