Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation

O. Seong Jun; Dahun Choi; Hyun Kim

doi:10.1109/ISOCC66390.2025.11329793

Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation

O. Seong Jun
, Dahun Choi
, Hyun Kim

Dept. of Electrical and Information Engineering

Seoul National University of Science and Technology (SNUST)

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

Diffusion models (DMs) excel in vision tasks but face high computational costs due to iterative denoising. While DeepCache improves efficiency via feature reuse, its static skip branch selection overlooks varying feature importance across stages. We propose a step-aware skip branch allocation method that dynamically optimizes the trade-off between inference speed and image quality. By allocating shallow branches to early and mid-stages and stage-specific branches to the late stage, the Speed-first strategy improves throughput by 30.4^%, and the Quality-first strategy reduces Fréchet Inception Distance (FID) by 29.5%, compared to the fixed skip branch 2 baseline. The method enables more efficient diffusion model inference.

Original language	English
Title of host publication	International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9798331586423
DOIs	https://doi.org/10.1109/ISOCC66390.2025.11329793
State	Published - 2025
Event	22nd International SoC Design Conference, ISOCC 2025 - Busan, Korea, Republic of Duration: 15 Oct 2025 → 18 Oct 2025

Publication series

Name	International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers

Conference

Conference	22nd International SoC Design Conference, ISOCC 2025
Country/Territory	Korea, Republic of
City	Busan
Period	15/10/25 → 18/10/25

Keywords

Computer Vision
Deep Learning
Diffusion Model
Skip Branch Allocation

Access to Document

10.1109/ISOCC66390.2025.11329793

Cite this

Seong Jun, O., Choi, D., & Kim, H. (2025). Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation. In International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers (International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISOCC66390.2025.11329793

@inproceedings{da459fe4a3de47229a07307e84e9b305,

title = "Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation",

abstract = "Diffusion models (DMs) excel in vision tasks but face high computational costs due to iterative denoising. While DeepCache improves efficiency via feature reuse, its static skip branch selection overlooks varying feature importance across stages. We propose a step-aware skip branch allocation method that dynamically optimizes the trade-off between inference speed and image quality. By allocating shallow branches to early and mid-stages and stage-specific branches to the late stage, the Speed-first strategy improves throughput by 30.4\%, and the Quality-first strategy reduces Fr{\'e}chet Inception Distance (FID) by 29.5\%, compared to the fixed skip branch 2 baseline. The method enables more efficient diffusion model inference.",

keywords = "Computer Vision, Deep Learning, Diffusion Model, Skip Branch Allocation",

author = "\{Seong Jun\}, O. and Dahun Choi and Hyun Kim",

note = "Publisher Copyright: {\textcopyright} 2025 IEEE.; 22nd International SoC Design Conference, ISOCC 2025 ; Conference date: 15-10-2025 Through 18-10-2025",

year = "2025",

doi = "10.1109/ISOCC66390.2025.11329793",

language = "English",

series = "International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers",

}

Seong Jun, O, Choi, D & Kim, H 2025, Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation. in International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers. International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers, Institute of Electrical and Electronics Engineers Inc., 22nd International SoC Design Conference, ISOCC 2025, Busan, Korea, Republic of, 15/10/25. https://doi.org/10.1109/ISOCC66390.2025.11329793

Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation. / Seong Jun, O.; Choi, Dahun; Kim, Hyun.
International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers. Institute of Electrical and Electronics Engineers Inc., 2025. (International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation

AU - Seong Jun, O.

AU - Choi, Dahun

AU - Kim, Hyun

PY - 2025

Y1 - 2025

N2 - Diffusion models (DMs) excel in vision tasks but face high computational costs due to iterative denoising. While DeepCache improves efficiency via feature reuse, its static skip branch selection overlooks varying feature importance across stages. We propose a step-aware skip branch allocation method that dynamically optimizes the trade-off between inference speed and image quality. By allocating shallow branches to early and mid-stages and stage-specific branches to the late stage, the Speed-first strategy improves throughput by 30.4%, and the Quality-first strategy reduces Fréchet Inception Distance (FID) by 29.5%, compared to the fixed skip branch 2 baseline. The method enables more efficient diffusion model inference.

AB - Diffusion models (DMs) excel in vision tasks but face high computational costs due to iterative denoising. While DeepCache improves efficiency via feature reuse, its static skip branch selection overlooks varying feature importance across stages. We propose a step-aware skip branch allocation method that dynamically optimizes the trade-off between inference speed and image quality. By allocating shallow branches to early and mid-stages and stage-specific branches to the late stage, the Speed-first strategy improves throughput by 30.4%, and the Quality-first strategy reduces Fréchet Inception Distance (FID) by 29.5%, compared to the fixed skip branch 2 baseline. The method enables more efficient diffusion model inference.

KW - Computer Vision

KW - Deep Learning

KW - Diffusion Model

KW - Skip Branch Allocation

UR - https://www.scopus.com/pages/publications/105033152855

U2 - 10.1109/ISOCC66390.2025.11329793

DO - 10.1109/ISOCC66390.2025.11329793

M3 - Conference contribution

AN - SCOPUS:105033152855

T3 - International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers

BT - International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 22nd International SoC Design Conference, ISOCC 2025

Y2 - 15 October 2025 through 18 October 2025

ER -

Seong Jun O, Choi D, Kim H. Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation. In International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers. Institute of Electrical and Electronics Engineers Inc. 2025. (International SoC Design Conference 2025, ISOCC 2025 - Proceedings of Technical Papers). doi: 10.1109/ISOCC66390.2025.11329793

Efficient Inference in Diffusion Models via Step-Aware Skip Branch Allocation

Abstract

Publication series

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this