Decentralized, cooperative and large-scale distributed storage systems that consist of a cluster of storage nodes attached with local disks can deliver high resource utilization, high availability and easy scalability. Therefore they become a cost-effective solution. High availability with node failures, strong data consistency, flexible expandability and high I/O throughput are still important challenges in the face of the system design. In this paper, we describe the design and implementation of a LAN-based decentralized storage cluster built upon off-the-shelf commodity components. It employs replication and Erasure-codes to provide high reliability and automatic failure recovery, adopts versioning to achieve single-file serializability and replicas consistency, supports data migration scheme to balance I/O workload and storage utilization, and provides global unified content cache to optimize memory-to-disk policy. We also present the experiment results to demonstrate the features and performance of our design.
Citation:
Bin Cai, Changsheng Xie, "High Reliability and Flexibility Storage Cluster in Decentralized Organization," iwnas, pp.194-202, 2006 International Workshop on Networking, Architecture, and Storages (IWNAS'06), 2006