Large Language Models in Biomedicine and Health: A Holistic Evaluation of the Effectiveness, Reliability and Ethics using Altmetrics

This study investigated the application of Large Language Models (LLMs), particularly ChatGPT and Bard, in biomedical and clinical sciences and health sciences via a combination of the altmetric approach and scientometric approach. This study analyzes Altmetric scores across various journals and FoRs, focusing on 942 publications since the launch of the ChatGPT in November 2022 (up to 13 November 2023) in Biomedical and Clinical Sciences and Health Sciences. Key findings highlight the growing impact of LLMs in the biomedical and health sciences, as evidenced by high Altmetric Attention Scores. Discussions revolve around ethical issues such as data privacy, AI biases and LLMs' role at the intersection of computational linguistics, AI and healthcare. The findings underscore the potential and challenges of LLMs in healthcare, emphasizing the need for enhanced accuracy, reliability and social acceptance. This study not only presents current trends and impacts but also provides key recommendations for advancing LLM technology, fostering interdisciplinary collaboration and establishing robust validation and regulatory frameworks to successfully integrate LLMs in biomedicine and health. These insights are crucial for guiding future advancements in healthcare research and practice.