Skip to main navigation Skip to search Skip to main content

특화 도메인 PDF 기반 멀티모달 RAG 질의응답 모델 설계 및 성능 분석

Research output: Contribution to journal › Article › peer-review

Abstract

As PDF-format technical documents become increasingly voluminous and complex in modern industrial environments, the demand for efficient methods of information retrieval and utilization continues to grow. Traditional keyword-based search approaches face limitations in accurately interpreting user intent and generating context-aware responses, especially in documents that integrate both textual and visual content.
To address these challenges, this study proposes a multimodal Retrieval-Augmented Generation (RAG) question-answering model tailored for domain-specific PDF documents.
Using vehicle maintenance manuals as a case study, we constructed a text-image integrated dataset and designed a multimodal RAG architecture combining a semantic similarity-based retriever with a LoRA fine-tuned generator.
Experimental results show that the proposed model outperforms baseline RAG models by 3.0 percentage points in BERTScore and cosine similarity, and by 18.0 percentage points in ROUGE-L. In a user evaluation with 20 participants, the model achieved an average satisfaction score of 4.4 out of 5.0. Notably, the provision of multimodal responses significantly improved users’ comprehension and information utilization.
This research demonstrates the effectiveness of the proposed approach for vehicle maintenance manuals and suggests potential applicability to technical documents with similar structural characteristics (hierarchical procedures, explicit image-text mappings). Further validation is required for broader domain generalization.
Original languageKorean
Pages (from-to)201-218
Number of pages18
Journal한국전자거래학회지
Volume24
Issue number3
StatePublished - Aug 2025

Keywords

  • PDF Document Processing
  • Multimodal RAG
  • Domain-Specific QA
  • Retrieval-Augmented Generation
  • Technical Document Automation

Cite this