Institutional Repository
| A distributed cache framework for metadata service of distributed file systems | |
| Sun, Yao (1); Liu, Jie (2); Ye, Dan (2); Zhong, Hua (2) | |
| 2013 | |
| Conference Name | 2013 19th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2013 |
| Pages | 51-58 |
| Conference Date | December 15, 2013 - December 18, 2013 |
| Conference Place | Seoul, Korea, Republic of |
| Indexed Type | EI |
| Publish Place | IEEE Computer Society |
| ISSN | 15219097 |
| ISBN | 9781479920815 |
| Department | (1) University of Chinese Academy of Sciences, Beijing, China; (2) Institute of Software, Chinese Academy of Sciences, Beijing, China |
| English Abstract | Most recent distributed file systems have adopted architecture with an independent metadata server cluster. However, potential multiple hotspots and flash crowds access patterns often cause a metadata service that violates performance Service Level Objectives. To maximize the throughput of the metadata service, an adaptive request load balancing framework is critical. We present a distributed cache framework above the distributed metadata management schemes to manage hotspots rather than managing all metadata to achieve request load balancing. This benefits the metadata hierarchical locality and the system scalability. Compared with data, metadata has its own distinct characteristics, such as small size and large quantity. The cost of useless metadata prefetching is much less than data prefetching. In light of this, we devise a time period-based prefetching strategy and a perfecting-based adaptive replacement cache algorithm to improve the performance of the distributed caching layer to adapt constantly changing workloads. Finally, we evaluate our approach with a hadoop distributed file system cluster. © 2013 IEEE.; Most recent distributed file systems have adopted architecture with an independent metadata server cluster. However, potential multiple hotspots and flash crowds access patterns often cause a metadata service that violates performance Service Level Objectives. To maximize the throughput of the metadata service, an adaptive request load balancing framework is critical. We present a distributed cache framework above the distributed metadata management schemes to manage hotspots rather than managing all metadata to achieve request load balancing. This benefits the metadata hierarchical locality and the system scalability. Compared with data, metadata has its own distinct characteristics, such as small size and large quantity. The cost of useless metadata prefetching is much less than data prefetching. In light of this, we devise a time period-based prefetching strategy and a perfecting-based adaptive replacement cache algorithm to improve the performance of the distributed caching layer to adapt constantly changing workloads. Finally, we evaluate our approach with a hadoop distributed file system cluster. © 2013 IEEE. |
| Language | 英语 |
| Content Type | 会议论文 |
| URI | http://ir.iscas.ac.cn/handle/311060/16686 |
| Collection | 中国科学院软件研究所 |
| Recommended Citation GB/T 7714 | Sun, Yao ,Liu, Jie ,Ye, Dan ,et al. A distributed cache framework for metadata service of distributed file systems[C]. IEEE Computer Society,2013:51-58. |
| Files in This Item: | There are no files associated with this item. | |||||
Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.
Edit Comment