Efficient and robust data integrity verification scheme for high-performance storage devices

Hwajung Kim; Inhwi Hwang; Heon Y. Yeom

doi:10.1145/3412841.3442113

Efficient and robust data integrity verification scheme for high-performance storage devices

Hwajung Kim, Inhwi Hwang, Heon Y. Yeom

Dept. of Information Technology and Media Engineering

Seoul National University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Scopus citations

Abstract

Most of the data generated on high-performance computing systems are transferred to storage in remote systems for various purposes such as backup. To detect data corruption caused by network or storage failures during data transfer, the receiver system verifies data integrity by comparing the checksum of the data. However, the internal operation of the storage device is not sufficiently investigated in the existing end-to-end integrity verification techniques. In this paper, we propose an efficient and robust data integrity verification scheme for large-scale data transfer between computing systems with high-performance storage devices. To ensure the robustness of the integrity verification, we control the order of I/O operations. In addition, we parallelize checksum computing and overlap it with I/O operations to make the integrity verification efficient.

Original language	English
Title of host publication	Proceedings of the 36th Annual ACM Symposium on Applied Computing, SAC 2021
Publisher	Association for Computing Machinery
Pages	1199-1202
Number of pages	4
ISBN (Electronic)	9781450381048
DOIs	https://doi.org/10.1145/3412841.3442113
State	Published - 22 Mar 2021
Event	36th Annual ACM Symposium on Applied Computing, SAC 2021 - Virtual, Online, Korea, Republic of Duration: 22 Mar 2021 → 26 Mar 2021

Publication series

Name	Proceedings of the ACM Symposium on Applied Computing

Conference

Conference	36th Annual ACM Symposium on Applied Computing, SAC 2021
Country/Territory	Korea, Republic of
City	Virtual, Online
Period	22/03/21 → 26/03/21

Keywords

data transfer
integrity verification
parallel execution
resource utilization
robustness

Access to Document

10.1145/3412841.3442113

Cite this

@inproceedings{7ea724dc9c7b42c68da1a716e147a783,

title = "Efficient and robust data integrity verification scheme for high-performance storage devices",

abstract = "Most of the data generated on high-performance computing systems are transferred to storage in remote systems for various purposes such as backup. To detect data corruption caused by network or storage failures during data transfer, the receiver system verifies data integrity by comparing the checksum of the data. However, the internal operation of the storage device is not sufficiently investigated in the existing end-to-end integrity verification techniques. In this paper, we propose an efficient and robust data integrity verification scheme for large-scale data transfer between computing systems with high-performance storage devices. To ensure the robustness of the integrity verification, we control the order of I/O operations. In addition, we parallelize checksum computing and overlap it with I/O operations to make the integrity verification efficient.",

keywords = "data transfer, integrity verification, parallel execution, resource utilization, robustness",

author = "Hwajung Kim and Inhwi Hwang and Yeom, \{Heon Y.\}",

note = "Publisher Copyright: {\textcopyright} 2021 Owner/Author.; 36th Annual ACM Symposium on Applied Computing, SAC 2021 ; Conference date: 22-03-2021 Through 26-03-2021",

year = "2021",

month = mar,

day = "22",

doi = "10.1145/3412841.3442113",

language = "English",

series = "Proceedings of the ACM Symposium on Applied Computing",

publisher = "Association for Computing Machinery",

pages = "1199--1202",

booktitle = "Proceedings of the 36th Annual ACM Symposium on Applied Computing, SAC 2021",

}

Kim, H, Hwang, I & Yeom, HY 2021, Efficient and robust data integrity verification scheme for high-performance storage devices. in Proceedings of the 36th Annual ACM Symposium on Applied Computing, SAC 2021. Proceedings of the ACM Symposium on Applied Computing, Association for Computing Machinery, pp. 1199-1202, 36th Annual ACM Symposium on Applied Computing, SAC 2021, Virtual, Online, Korea, Republic of, 22/03/21. https://doi.org/10.1145/3412841.3442113

Efficient and robust data integrity verification scheme for high-performance storage devices. / Kim, Hwajung; Hwang, Inhwi; Yeom, Heon Y.
Proceedings of the 36th Annual ACM Symposium on Applied Computing, SAC 2021. Association for Computing Machinery, 2021. p. 1199-1202 (Proceedings of the ACM Symposium on Applied Computing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient and robust data integrity verification scheme for high-performance storage devices

AU - Kim, Hwajung

AU - Hwang, Inhwi

AU - Yeom, Heon Y.

PY - 2021/3/22

Y1 - 2021/3/22

N2 - Most of the data generated on high-performance computing systems are transferred to storage in remote systems for various purposes such as backup. To detect data corruption caused by network or storage failures during data transfer, the receiver system verifies data integrity by comparing the checksum of the data. However, the internal operation of the storage device is not sufficiently investigated in the existing end-to-end integrity verification techniques. In this paper, we propose an efficient and robust data integrity verification scheme for large-scale data transfer between computing systems with high-performance storage devices. To ensure the robustness of the integrity verification, we control the order of I/O operations. In addition, we parallelize checksum computing and overlap it with I/O operations to make the integrity verification efficient.

AB - Most of the data generated on high-performance computing systems are transferred to storage in remote systems for various purposes such as backup. To detect data corruption caused by network or storage failures during data transfer, the receiver system verifies data integrity by comparing the checksum of the data. However, the internal operation of the storage device is not sufficiently investigated in the existing end-to-end integrity verification techniques. In this paper, we propose an efficient and robust data integrity verification scheme for large-scale data transfer between computing systems with high-performance storage devices. To ensure the robustness of the integrity verification, we control the order of I/O operations. In addition, we parallelize checksum computing and overlap it with I/O operations to make the integrity verification efficient.

KW - data transfer

KW - integrity verification

KW - parallel execution

KW - resource utilization

KW - robustness

UR - http://www.scopus.com/inward/record.url?scp=85104992816&partnerID=8YFLogxK

U2 - 10.1145/3412841.3442113

DO - 10.1145/3412841.3442113

M3 - Conference contribution

AN - SCOPUS:85104992816

T3 - Proceedings of the ACM Symposium on Applied Computing

SP - 1199

EP - 1202

BT - Proceedings of the 36th Annual ACM Symposium on Applied Computing, SAC 2021

PB - Association for Computing Machinery

T2 - 36th Annual ACM Symposium on Applied Computing, SAC 2021

Y2 - 22 March 2021 through 26 March 2021

ER -

Efficient and robust data integrity verification scheme for high-performance storage devices

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this