English
Russian
Chatbots based on artificial intelligence in oncourology: assessment of the reliability and quality of medical information DOI: 10.29188/2222-8543-2025-18-2-30-36
- Arutyunyan P.A. – urologist, oncologist, researcher at the S.P. Botkin Moscow Multidisciplinary Scientific and Clinical Center of the Moscow Health Department, leading specialist in urology at the Research Institute for Health Organization and Medical Management of the Moscow Health Department, postgraduate student at the Department of Urology at the Russian University of Medicine of the Russian Ministry of Health, Moscow, Russia; RSCI Author ID 1193661, https://orcid.org/0000-0002-6228-012X
- Vasilyev A.O. – PhD, urologist, oncologist at the Moscow Urology Center of the S.P. Botkin Multidisciplinary Scientific and Clinical Center of the Moscow Health Department, leading specialist in urology at the Research Institute for Health Organization and Medical Management of the Moscow Health Department, assistant at the Department of Urology at the Russian University of Medicine of the Russian Ministry of Health, Moscow, Russia; RSCI Author ID 832619, https://orcid.org/0000-0001-5468-0011
- Govorov A.V. – Dr. Sci., Head of the Oncourology Department of the Moscow Urology Center of the S.P. Botkin Multidisciplinary Scientific and Clinical Center of the Moscow Health Department, Professor of the Urology Department of the Russian University of Medicine of the Ministry of Health of the Russian Federation, Moscow, Russia; RSCI Author ID 711844, https://orcid.org/0000-0003-3299-0574
- Kim Yu.A. – PhD, physician-methodologist of the Moscow Territorial Scientific and Practical Center for Disaster Medicine (TSEMP) of the Moscow Health Department, Moscow, Russia; RSCI Author ID 1068822, https://orcid.org/0000-0001-6390-3408
- Medvedev F.A. – resident physician of the Department of Urology of the Russian University of Medicine of the Ministry of Health of the Russian Federation, analyst of the OMD on urology of the Research Institute of Healthcare Organization and Medical Management of the Department of Health of the Ministry of Health of the Russian Federation, Moscow, Russia; RSCI Author ID 1288228, https://orcid.org/0000-0003-0460-4816
- Khodyreva L.A. – Dr. Sci., Head of the OMD on Urology of the Research Institute of Healthcare Organization and Medical Management, Moscow Health Department, Moscow, Russia; RSCI Author ID 467082, https://orcid.org/0000-0002-0751-4982
- Pushkar D. Yu. – Dr. Sci., Professor, Academician of the Russian Academy of Sciences, Chief Urologist of the Russian Ministry of Health, Head of the Urology Department of the Russian University of Medicine of the Russian Ministry of Health, Head of the Moscow Urology Center of the S.P. Botkin Moscow Multidisciplinary Scientific and Clinical Center of the Moscow Department of Health, Moscow, Russia; RSCI Author ID 417122, https://orcid.org/0000-0002-6096-5723
358 Introduction. In the era of digital progress and the development of artificial intelligence (AI) technologies, the potential application of chatbots in various fields, including medicine, has gained significant attention. With the global accessibility of the internet and the growing popularity of such services, there is an increasing need to evaluate the reliability and quality of the data generated, particularly concerning malignant neoplasms.
The aim of this study was to analyze the quality of medical information related to prostate and bladder cancer using AI-based chatbots.
Materials and methods. A study was conducted involving the examination and analysis of web traffic data from StatCounter. Based on the results of the web analysis to determine the leading search engine in overall web traffic in Russia, we used data from the Yandex Wordstat analytics service. We also conducted an assessment of responses from four AI-based chatbots to the most in-demand medical queries related to the two urological cancers with the highest incidence rates-prostate cancer and bladder cancer. We used publicly available versions of the four AI-based chatbots: ChatGPT, Perplexity, YaGPT, and GigaChat. To evaluate the quality of the medical information provided by the chatbots, we applied the validated DIS- CERN assessment tool. In addition, qualitative analysis and expert evaluation using DISCERN were carried out with the participation of 50 respondents-oncourologists with at least 10 years of professional experience in Russia.
Results. We obtained and analyzed the evaluation results using the DISCERN scale. The highest overall score for medical information quality was achieved by the Perplexity chatbot, followed by ChatGPT in second place, and GigaChat in third. YaGPT ranked fourth and last. It is worth noting that all chatbots, to varying degrees, made errors and inaccuracies in generating medical information in response to the relevant queries.
Conclusion. Today, AI-based chatbots represent a promising and in-demand direction. With the advancement of technology and the widespread adoption of software solutions, more people are turning to such services in search of authoritative medical information. However, at this stage, not all chatbots are capable of providing accurate medical content. Further development of AI architecture and internal algorithms is required, as well as the creation of standards and recommendations for integrating chatbots into medical information systems. This will help shape the direction of AI solutions for the coming decades and support both physicians and patients in diagnostics and decision-making.
| Attachment | Size |
|---|---|
| Download | 1.3 MB |
Other issue articles
