A distributed cache framework for metadata service of distributed file systems

ISCAS OpenIR

	A distributed cache framework for metadata service of distributed file systems
	Sun, Yao (1); Liu, Jie (2); Ye, Dan (2); Zhong, Hua (2)
	2013
会议名称	2013 19th IEEE International Conference on Parallel and Distributed Systems, ICPADS 2013
页码	51-58
会议日期	December 15, 2013 - December 18, 2013
会议地点	Seoul, Korea, Republic of
收录类别	EI
出版地	IEEE Computer Society
ISSN	15219097
ISBN	9781479920815
部门归属	(1) University of Chinese Academy of Sciences, Beijing, China; (2) Institute of Software, Chinese Academy of Sciences, Beijing, China
摘要	Most recent distributed file systems have adopted architecture with an independent metadata server cluster. However, potential multiple hotspots and flash crowds access patterns often cause a metadata service that violates performance Service Level Objectives. To maximize the throughput of the metadata service, an adaptive request load balancing framework is critical. We present a distributed cache framework above the distributed metadata management schemes to manage hotspots rather than managing all metadata to achieve request load balancing. This benefits the metadata hierarchical locality and the system scalability. Compared with data, metadata has its own distinct characteristics, such as small size and large quantity. The cost of useless metadata prefetching is much less than data prefetching. In light of this, we devise a time period-based prefetching strategy and a perfecting-based adaptive replacement cache algorithm to improve the performance of the distributed caching layer to adapt constantly changing workloads. Finally, we evaluate our approach with a hadoop distributed file system cluster. © 2013 IEEE.; Most recent distributed file systems have adopted architecture with an independent metadata server cluster. However, potential multiple hotspots and flash crowds access patterns often cause a metadata service that violates performance Service Level Objectives. To maximize the throughput of the metadata service, an adaptive request load balancing framework is critical. We present a distributed cache framework above the distributed metadata management schemes to manage hotspots rather than managing all metadata to achieve request load balancing. This benefits the metadata hierarchical locality and the system scalability. Compared with data, metadata has its own distinct characteristics, such as small size and large quantity. The cost of useless metadata prefetching is much less than data prefetching. In light of this, we devise a time period-based prefetching strategy and a perfecting-based adaptive replacement cache algorithm to improve the performance of the distributed caching layer to adapt constantly changing workloads. Finally, we evaluate our approach with a hadoop distributed file system cluster. © 2013 IEEE.
语种	英语
内容类型	会议论文
URI标识	http://ir.iscas.ac.cn/handle/311060/16686
专题	中国科学院软件研究所
推荐引用方式 GB/T 7714	Sun, Yao ,Liu, Jie ,Ye, Dan ,et al. A distributed cache framework for metadata service of distributed file systems[C]. IEEE Computer Society,2013:51-58.