Compression accelerator for hadoop appliance

Sang Don Kim, Seong Mo Lee, Sang Muk Lee, Ji Hoon Jang, Jae Gi Son, Young Hwan Kim, Seung Eun Lee

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

7 Scopus citations

Abstract

In this paper, we propose an accelerator for Hadoop appliances. Data servers receive significant data traffic because of several services based on networks. Processing such big data requires sufficient communication bandwidth for big data management. In order to increase communication bandwidth, compression algorithms are adopted in Hadoop appliances, although additional computation overhead is required. Our accelerator compresses data from a PCIe (Peripheral Component Interconnect Express) interface, thus reducing the size of data that should be transmitted through a network. As a result, computation overhead of the main processor in a server is decreased and communication bandwidth is increased.

Original languageEnglish
Title of host publicationInternet of Vehicles - Technologies and Services - First International Conference, IOV 2014, Proceedings
PublisherSpringer Verlag
Pages416-423
Number of pages8
ISBN (Print)9783319111667
DOIs
StatePublished - 2014
Event1st International Conference on Internet of Vehicles, IOV 2014 - Beijing, China
Duration: 1 Sep 20143 Sep 2014

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume8662 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference1st International Conference on Internet of Vehicles, IOV 2014
Country/TerritoryChina
CityBeijing
Period1/09/143/09/14

Keywords

  • Compression Accelerator
  • FPGA
  • Hadoop
  • LZ4
  • PCI Express

Fingerprint

Dive into the research topics of 'Compression accelerator for hadoop appliance'. Together they form a unique fingerprint.

Cite this