TY - GEN
T1 - ScaleDFS
T2 - 33rd International Symposium on High-Performance Parallel and Distributed Computing, HPDC 2024
AU - Song, Mansub
AU - Nguyen, Lan Anh
AU - Kim, Sunggon
AU - Eom, Hyeonsang
AU - Son, Yongseok
N1 - Publisher Copyright:
© 2024 held by the owner/author(s).
PY - 2024/6/3
Y1 - 2024/6/3
N2 - This paper introduces a novel file system, ScaleDFS, designed to accelerate decentralized file sharing for a private network, leveraging the potential of scaling file management based on directed acyclic graph (DAG) on modern hardware. Specifically, in ScaleDFS, we first design a DAG builder that parallelizes the construction of DAG nodes for a file while preserving critical orders to speed up the uploading process. Second, we introduce a DAG reader that retrieves leaf DAG nodes in parallel without graph traversal assisted by a devised DAG cache to accelerate the downloading process. Finally, we present a DAG remover that rapidly identifies obsolete DAG nodes/data and removes them in parallel to mitigate the garbage collection overhead without compromising consistency. We implement ScaleDFS based on IPFS and demonstrate that ScaleDFS outperforms IPFS by up to 3.7×, 1.8×, and 12.6× in realistic file, private blockchain, and gateway workloads, respectively.
AB - This paper introduces a novel file system, ScaleDFS, designed to accelerate decentralized file sharing for a private network, leveraging the potential of scaling file management based on directed acyclic graph (DAG) on modern hardware. Specifically, in ScaleDFS, we first design a DAG builder that parallelizes the construction of DAG nodes for a file while preserving critical orders to speed up the uploading process. Second, we introduce a DAG reader that retrieves leaf DAG nodes in parallel without graph traversal assisted by a devised DAG cache to accelerate the downloading process. Finally, we present a DAG remover that rapidly identifies obsolete DAG nodes/data and removes them in parallel to mitigate the garbage collection overhead without compromising consistency. We implement ScaleDFS based on IPFS and demonstrate that ScaleDFS outperforms IPFS by up to 3.7×, 1.8×, and 12.6× in realistic file, private blockchain, and gateway workloads, respectively.
KW - decentralized file system
KW - merkle-DAG
KW - private network
UR - https://www.scopus.com/pages/publications/85204935335
U2 - 10.1145/3625549.3658690
DO - 10.1145/3625549.3658690
M3 - Conference contribution
AN - SCOPUS:85204935335
T3 - HPDC 2024 - Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing
SP - 295
EP - 308
BT - HPDC 2024 - Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing
PB - Association for Computing Machinery, Inc
Y2 - 3 June 2024 through 7 June 2024
ER -