Home // AISyS 2024, The First International Conference on AI-based Systems and Services // View article

Combining Templates and Language Models for the Automatic Creation of Scientific Overviews

Authors:
Sarah Frank
Andreas Wagner
Christian Gütl

Keywords: automatic summarization; hybrid summarization; language models; natural language processing; templates

Abstract:
The number of scientific publications is increasing at a rate that makes them progressively more impossible to keep up with. Consequently, automatic creation of summaries from a collection of articles could significantly speed up the selection of publications of interest. This paper focuses on the use case of ultra-short summaries to be used for the creation of topic overviews, as often found in journal editorials. We used a combination of a pre-trained language model and templates to create a coherent text summarizing the papers contained within single journal issues. Following this, we conducted two user studies. The results were generally promising, with users preferring the automatically created summary in a majority of cases. Evaluations of the accuracy, coverage, fluency, and informativeness of the summaries showed that most users found them to be good. However, the variation in the evaluation scores was significant both by user and summary. Text quality was shown to be graded differently according to the user's requirements and familiarity with the typical form of this kind of summary. Furthermore, the importance of high-quality base summaries from the language model, as well as a high number of available templates, cannot be overstated.

Pages: 46 to 51

Copyright: Copyright (c) IARIA, 2024

Publication date: September 29, 2024

Published in: conference

ISBN: 978-1-68558-192-3

Location: Venice, Italy

Dates: from September 29, 2024 to October 3, 2024