Improving the Summarization Effectiveness of Abstractive Datasets through Contrastive Learning

Most studies on abstractive summarization are conducted in a supervised learning framework, aiming to generate a golden summary from the original document. In this process, the model focuses on portions of the document that closely resemble the golden summary to produce a coherent output. Consequently, current methodologies tend to achieve higher performance on extractive datasets compared to abstractive datasets, indicating diminished effectiveness on more abstracted content. To address this, our study proposes a methodology that maintains high effectiveness on abstractive datasets. Specifically, we introduce a multi-task learning approach that incorporates both salient and non-salient information during training. This is implemented by adding a contrastive objective to the fine-tuning phase of an encoder-decoder language model. Salient and non-salient parts are selected based on ROUGE-L F1 scores, and their relationships are learned through a triplet loss function. The proposed method is evaluated on five benchmark summarization datasets, including two extractive and three abstractive datasets. Experimental results demonstrate significant performance improvements on abstractive datasets, particularly those with high levels of abstraction, compared to existing abstractive summarization methods.