Abstract
As PDF-format technical documents become increasingly voluminous and complex in modern industrial environments, the demand for efficient methods of information retrieval and utilization continues to grow. Traditional keyword-based search approaches face limitations in accurately interpreting user intent and generating context-aware responses, especially in documents that integrate both textual and visual content.
To address these challenges, this study proposes a multimodal Retrieval-Augmented Generation (RAG) question-answering model tailored for domain-specific PDF documents.
Using vehicle maintenance manuals as a case study, we constructed a text-image integrated dataset and designed a multimodal RAG architecture combining a semantic similarity-based retriever with a LoRA fine-tuned generator.
Experimental results show that the proposed model outperforms baseline RAG models by 3.0 percentage points in BERTScore and cosine similarity, and by 18.0 percentage points in ROUGE-L. In a user evaluation with 20 participants, the model achieved an average satisfaction score of 4.4 out of 5.0. Notably, the provision of multimodal responses significantly improved users’ comprehension and information utilization.
This research demonstrates the effectiveness of the proposed approach for vehicle maintenance manuals and suggests potential applicability to technical documents with similar structural characteristics (hierarchical procedures, explicit image-text mappings). Further validation is required for broader domain generalization.
To address these challenges, this study proposes a multimodal Retrieval-Augmented Generation (RAG) question-answering model tailored for domain-specific PDF documents.
Using vehicle maintenance manuals as a case study, we constructed a text-image integrated dataset and designed a multimodal RAG architecture combining a semantic similarity-based retriever with a LoRA fine-tuned generator.
Experimental results show that the proposed model outperforms baseline RAG models by 3.0 percentage points in BERTScore and cosine similarity, and by 18.0 percentage points in ROUGE-L. In a user evaluation with 20 participants, the model achieved an average satisfaction score of 4.4 out of 5.0. Notably, the provision of multimodal responses significantly improved users’ comprehension and information utilization.
This research demonstrates the effectiveness of the proposed approach for vehicle maintenance manuals and suggests potential applicability to technical documents with similar structural characteristics (hierarchical procedures, explicit image-text mappings). Further validation is required for broader domain generalization.
| Original language | Korean |
|---|---|
| Pages (from-to) | 201-218 |
| Number of pages | 18 |
| Journal | 한국전자거래학회지 |
| Volume | 24 |
| Issue number | 3 |
| State | Published - Aug 2025 |
Keywords
- PDF Document Processing
- Multimodal RAG
- Domain-Specific QA
- Retrieval-Augmented Generation
- Technical Document Automation
Cite this
- APA
- Author
- BIBTEX
- Harvard
- Standard
- RIS
- Vancouver