Quartile Prediction and Journal Recommendation Using Deep Learning Models for Artificial Intelligence Articles

Journal recommendation systems serve as valuable tools for researchers, addressing the complex task of multi-class text classification. With the advent of Transformer architectures, there is newfound potential to enhance existing recommendation systems, particularly in the realm of academic journals. While current technologies are capable of classifying journals based on article content, we still lack an algorithm that can predict the quartile ranking of journals. Such a development would be immensely beneficial for researchers to assess their articles before submission. In our study, we tackle both tasks simultaneously. We trained various state-of-the-art Transformer architectures and machine learning algorithms, ranging from BERT to GPT-2. Surprisingly, we achieved better quantitative results with smaller models, especially DistilBERT, as well as classical classifiers. However, when it came to quartile prediction, success was limited to testing within the same journals. Generalization across different journals proved elusive. This observation strongly suggests that quartile prediction currently relies indirectly on journal classification, highlighting the limitations of existing technology, the collected dataset, or the impossibility of solving the task.