A 40nm 5.6TOPS/W 239GOPS/mm2Self-Attention Processor with Sign Random Projection-based Approximation

Seong Hoon Seo; Soosung Kim; Sung Jun Jung; Sangwoo Kwon; Hyunseung Lee; Jae W. Lee

doi:10.1109/ESSCIRC55480.2022.9911343

A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation

Seong Hoon Seo, Soosung Kim, Sung Jun Jung, Sangwoo Kwon, Hyunseung Lee, Jae W. Lee

Dept. of Mechanical System and Design Engineering

Seoul National University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Transformer architecture is one of the most remarkable recent breakthroughs in neural networks, achieving state-of-The-Art (SOTA) performance on various natural language processing (NLP) and computer vision tasks. Self-Attention is the key enabling operation for transformer-based models. However, its quadratic computational complexity to the sequence length makes this operation the major performance bottleneck for those models. Thus, we propose a novel self-Attention accelerator that skips most of the computation by utilizing an approximate candidate selection algorithm. Implemented in a 40nm CMOS technology, our 5.64 mm2 chip operates at 100-600 MHz consuming 48.3-685 mW to achieve the energy and area efficiency of 0.354-5.61 TOPS/W and 239 GOPS/mm2, respectively.

Original language	English
Title of host publication	ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	85-88
Number of pages	4
ISBN (Electronic)	9781665484947
DOIs	https://doi.org/10.1109/ESSCIRC55480.2022.9911343
State	Published - 2022
Event	48th IEEE European Solid State Circuits Conference, ESSCIRC 2022 - Milan, Italy Duration: 19 Sep 2022 → 22 Sep 2022

Publication series

Name	ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

Conference

Conference	48th IEEE European Solid State Circuits Conference, ESSCIRC 2022
Country/Territory	Italy
City	Milan
Period	19/09/22 → 22/09/22

Access to Document

10.1109/ESSCIRC55480.2022.9911343

Cite this

Seo, S. H., Kim, S., Jung, S. J., Kwon, S., Lee, H., & Lee, J. W. (2022). A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation. In ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings (pp. 85-88). (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ESSCIRC55480.2022.9911343

Seo, Seong Hoon ; Kim, Soosung ; Jung, Sung Jun et al. / A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation. ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. pp. 85-88 (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings).

@inproceedings{2e64a75f796c49c690ab663e97830d7d,

title = "A 40nm 5.6TOPS/W 239GOPS/mm2Self-Attention Processor with Sign Random Projection-based Approximation",

abstract = "Transformer architecture is one of the most remarkable recent breakthroughs in neural networks, achieving state-of-The-Art (SOTA) performance on various natural language processing (NLP) and computer vision tasks. Self-Attention is the key enabling operation for transformer-based models. However, its quadratic computational complexity to the sequence length makes this operation the major performance bottleneck for those models. Thus, we propose a novel self-Attention accelerator that skips most of the computation by utilizing an approximate candidate selection algorithm. Implemented in a 40nm CMOS technology, our 5.64 mm2 chip operates at 100-600 MHz consuming 48.3-685 mW to achieve the energy and area efficiency of 0.354-5.61 TOPS/W and 239 GOPS/mm2, respectively.",

author = "Seo, \{Seong Hoon\} and Soosung Kim and Jung, \{Sung Jun\} and Sangwoo Kwon and Hyunseung Lee and Lee, \{Jae W.\}",

note = "Publisher Copyright: {\textcopyright} 2022 IEEE.; 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022 ; Conference date: 19-09-2022 Through 22-09-2022",

year = "2022",

doi = "10.1109/ESSCIRC55480.2022.9911343",

language = "English",

series = "ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "85--88",

booktitle = "ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings",

}

Seo, SH, Kim, S, Jung, SJ, Kwon, S, Lee, H & Lee, JW 2022, A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation. in ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 85-88, 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022, Milan, Italy, 19/09/22. https://doi.org/10.1109/ESSCIRC55480.2022.9911343

A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation. / Seo, Seong Hoon; Kim, Soosung; Jung, Sung Jun et al.
ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc., 2022. p. 85-88 (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - A 40nm 5.6TOPS/W 239GOPS/mm2Self-Attention Processor with Sign Random Projection-based Approximation

AU - Seo, Seong Hoon

AU - Kim, Soosung

AU - Jung, Sung Jun

AU - Kwon, Sangwoo

AU - Lee, Hyunseung

AU - Lee, Jae W.

PY - 2022

Y1 - 2022

N2 - Transformer architecture is one of the most remarkable recent breakthroughs in neural networks, achieving state-of-The-Art (SOTA) performance on various natural language processing (NLP) and computer vision tasks. Self-Attention is the key enabling operation for transformer-based models. However, its quadratic computational complexity to the sequence length makes this operation the major performance bottleneck for those models. Thus, we propose a novel self-Attention accelerator that skips most of the computation by utilizing an approximate candidate selection algorithm. Implemented in a 40nm CMOS technology, our 5.64 mm2 chip operates at 100-600 MHz consuming 48.3-685 mW to achieve the energy and area efficiency of 0.354-5.61 TOPS/W and 239 GOPS/mm2, respectively.

AB - Transformer architecture is one of the most remarkable recent breakthroughs in neural networks, achieving state-of-The-Art (SOTA) performance on various natural language processing (NLP) and computer vision tasks. Self-Attention is the key enabling operation for transformer-based models. However, its quadratic computational complexity to the sequence length makes this operation the major performance bottleneck for those models. Thus, we propose a novel self-Attention accelerator that skips most of the computation by utilizing an approximate candidate selection algorithm. Implemented in a 40nm CMOS technology, our 5.64 mm2 chip operates at 100-600 MHz consuming 48.3-685 mW to achieve the energy and area efficiency of 0.354-5.61 TOPS/W and 239 GOPS/mm2, respectively.

UR - http://www.scopus.com/inward/record.url?scp=85141465342&partnerID=8YFLogxK

U2 - 10.1109/ESSCIRC55480.2022.9911343

DO - 10.1109/ESSCIRC55480.2022.9911343

M3 - Conference contribution

AN - SCOPUS:85141465342

T3 - ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

SP - 85

EP - 88

BT - ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 48th IEEE European Solid State Circuits Conference, ESSCIRC 2022

Y2 - 19 September 2022 through 22 September 2022

ER -

Seo SH, Kim S, Jung SJ, Kwon S, Lee H, Lee JW. A 40nm 5.6TOPS/W 239GOPS/mm²Self-Attention Processor with Sign Random Projection-based Approximation. In ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings. Institute of Electrical and Electronics Engineers Inc. 2022. p. 85-88. (ESSCIRC 2022 - IEEE 48th European Solid State Circuits Conference, Proceedings). doi: 10.1109/ESSCIRC55480.2022.9911343