Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data

Jae Yeop Jeong; Yeong Gi Hong; Sumin Hong; Ji Yeon Oh; Yuchul Jung; Sang Ho Kim; Jin Woo Jeong

doi:10.1007/978-3-031-25075-0_5

Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data

Jae Yeop Jeong, Yeong Gi Hong, Sumin Hong, Ji Yeon Oh, Yuchul Jung, Sang Ho Kim, Jin Woo Jeong

Dept. of Industrial Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Scopus citations

Abstract

Facial expression recognition in-the-wild is essential for various interactive computing applications. Especially, “Learning from Synthetic Data” is an important topic in the facial expression recognition task. In this paper, we propose a multi-task learning-based facial expression recognition approach where emotion and appearance perspectives of facial images are jointly learned. We also present our experimental results on validation and test set of the LSD challenge introduced in the 4th affective behavior analysis in-the-wild competition. Our method achieved the mean F1 score of 71.82 on the validation and 35.87 on the test set, ranking third place on the final leaderboard.

Original language	English
Title of host publication	Computer Vision – ECCV 2022 Workshops, Proceedings
Editors	Leonid Karlinsky, Tomer Michaeli, Ko Nishino
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	60-75
Number of pages	16
ISBN (Print)	9783031250743
DOIs	https://doi.org/10.1007/978-3-031-25075-0_5
State	Published - 2023
Event	17th European Conference on Computer Vision, ECCV 2022 - Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022

Publication series

Name	Lecture Notes in Computer Science
Volume	13806 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	17th European Conference on Computer Vision, ECCV 2022
Country/Territory	Israel
City	Tel Aviv
Period	23/10/22 → 27/10/22

Keywords

Ensemble approach
Facial expression recognition
Leaning from synthetic data
Multi-task learning

Access to Document

10.1007/978-3-031-25075-0_5

Cite this

Jeong, J. Y., Hong, Y. G., Hong, S., Oh, J. Y., Jung, Y., Kim, S. H., & Jeong, J. W. (2023). Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data. In L. Karlinsky, T. Michaeli, & K. Nishino (Eds.), Computer Vision – ECCV 2022 Workshops, Proceedings (pp. 60-75). (Lecture Notes in Computer Science; Vol. 13806 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-031-25075-0_5

Jeong, Jae Yeop ; Hong, Yeong Gi ; Hong, Sumin et al. / Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data. Computer Vision – ECCV 2022 Workshops, Proceedings. editor / Leonid Karlinsky ; Tomer Michaeli ; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. pp. 60-75 (Lecture Notes in Computer Science).

@inproceedings{081a0b641daf4cc3940d22b52d08c652,

title = "Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data",

abstract = "Facial expression recognition in-the-wild is essential for various interactive computing applications. Especially, “Learning from Synthetic Data” is an important topic in the facial expression recognition task. In this paper, we propose a multi-task learning-based facial expression recognition approach where emotion and appearance perspectives of facial images are jointly learned. We also present our experimental results on validation and test set of the LSD challenge introduced in the 4th affective behavior analysis in-the-wild competition. Our method achieved the mean F1 score of 71.82 on the validation and 35.87 on the test set, ranking third place on the final leaderboard.",

keywords = "Ensemble approach, Facial expression recognition, Leaning from synthetic data, Multi-task learning",

author = "Jeong, \{Jae Yeop\} and Hong, \{Yeong Gi\} and Sumin Hong and Oh, \{Ji Yeon\} and Yuchul Jung and Kim, \{Sang Ho\} and Jeong, \{Jin Woo\}",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 17th European Conference on Computer Vision, ECCV 2022 ; Conference date: 23-10-2022 Through 27-10-2022",

year = "2023",

doi = "10.1007/978-3-031-25075-0\_5",

language = "English",

isbn = "9783031250743",

series = "Lecture Notes in Computer Science",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "60--75",

editor = "Leonid Karlinsky and Tomer Michaeli and Ko Nishino",

booktitle = "Computer Vision – ECCV 2022 Workshops, Proceedings",

}

Jeong, JY, Hong, YG, Hong, S, Oh, JY, Jung, Y, Kim, SH & Jeong, JW 2023, Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data. in L Karlinsky, T Michaeli & K Nishino (eds), Computer Vision – ECCV 2022 Workshops, Proceedings. Lecture Notes in Computer Science, vol. 13806 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 60-75, 17th European Conference on Computer Vision, ECCV 2022, Tel Aviv, Israel, 23/10/22. https://doi.org/10.1007/978-3-031-25075-0_5

Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data. / Jeong, Jae Yeop; Hong, Yeong Gi; Hong, Sumin et al.
Computer Vision – ECCV 2022 Workshops, Proceedings. ed. / Leonid Karlinsky; Tomer Michaeli; Ko Nishino. Springer Science and Business Media Deutschland GmbH, 2023. p. 60-75 (Lecture Notes in Computer Science; Vol. 13806 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data

AU - Jeong, Jae Yeop

AU - Hong, Yeong Gi

AU - Hong, Sumin

AU - Oh, Ji Yeon

AU - Jung, Yuchul

AU - Kim, Sang Ho

AU - Jeong, Jin Woo

PY - 2023

Y1 - 2023

N2 - Facial expression recognition in-the-wild is essential for various interactive computing applications. Especially, “Learning from Synthetic Data” is an important topic in the facial expression recognition task. In this paper, we propose a multi-task learning-based facial expression recognition approach where emotion and appearance perspectives of facial images are jointly learned. We also present our experimental results on validation and test set of the LSD challenge introduced in the 4th affective behavior analysis in-the-wild competition. Our method achieved the mean F1 score of 71.82 on the validation and 35.87 on the test set, ranking third place on the final leaderboard.

AB - Facial expression recognition in-the-wild is essential for various interactive computing applications. Especially, “Learning from Synthetic Data” is an important topic in the facial expression recognition task. In this paper, we propose a multi-task learning-based facial expression recognition approach where emotion and appearance perspectives of facial images are jointly learned. We also present our experimental results on validation and test set of the LSD challenge introduced in the 4th affective behavior analysis in-the-wild competition. Our method achieved the mean F1 score of 71.82 on the validation and 35.87 on the test set, ranking third place on the final leaderboard.

KW - Ensemble approach

KW - Facial expression recognition

KW - Leaning from synthetic data

KW - Multi-task learning

UR - http://www.scopus.com/inward/record.url?scp=85150984726&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-25075-0_5

DO - 10.1007/978-3-031-25075-0_5

M3 - Conference contribution

AN - SCOPUS:85150984726

SN - 9783031250743

T3 - Lecture Notes in Computer Science

SP - 60

EP - 75

BT - Computer Vision – ECCV 2022 Workshops, Proceedings

A2 - Karlinsky, Leonid

A2 - Michaeli, Tomer

A2 - Nishino, Ko

PB - Springer Science and Business Media Deutschland GmbH

T2 - 17th European Conference on Computer Vision, ECCV 2022

Y2 - 23 October 2022 through 27 October 2022

ER -

Jeong JY, Hong YG, Hong S, Oh JY, Jung Y, Kim SH et al. Ensemble of Multi-task Learning Networks for Facial Expression Recognition In-the-Wild with Learning from Synthetic Data. In Karlinsky L, Michaeli T, Nishino K, editors, Computer Vision – ECCV 2022 Workshops, Proceedings. Springer Science and Business Media Deutschland GmbH. 2023. p. 60-75. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-25075-0_5