Interpretable Link Prediction in AI-Driven Cancer Research: Uncovering Co-Authorship Patterns

Artificial Intelligence (AI) is reshaping cancer diagnosis and treatment, yet the formation of effective interdisciplinary research teams remains a major challenge. This study analyzes 7,738 publications (2000-2017) from Scopus to investigate collaboration dynamics in AI-driven cancer research. We constructed 36 overlapping co-authorship networks representing new, persistent, and discontinued collaborations. Using both attribute-based and structure-based features, we developed and compared four machine learning classifiers-logistic regression, decision tree, random forest, and XGBoost. Model interpretability was achieved using Shapley Additive Explanations (SHAP) to identify the most influential factors driving each collaboration pattern. The random forest model achieved the highest recall for all collaboration types. The discipline similarity score consistently promoted new and persistent collaborations while negatively affecting discontinued ones, whereas high productivity and seniority were associated with collaboration discontinuation. These insights can inform research policy, guiding institutions and funding agencies in designing strategies that foster sustainable and interdisciplinary research collaborations. While the study provides a robust data-driven framework for understanding co-authorship dynamics, it is limited by the assumption that co-authorship equates to collaboration, which may not capture all forms of scientific cooperation.