In a recent study published in the Nature journal Hypertension Research, researchers examined the responses given by the artificial intelligence tools ChatGPT and Bing to questions about blood pressure monitoring methods at home.
They compared them for completeness, accuracy, and reproducibility to determine how reliable these tools were in providing knowledge about medical conditions.
Study: Comparing ChatGPT and Bing, in response to the Home Blood Pressure Monitoring (HBPM) knowledge checklist. Image Credit: SomYuZu/Shutterstock.com
Background
Hypertension or high blood pressure is a highly prevalent global health problem that also increases the risk of other diseases such as cardiovascular disease, kidney disorder, neurological problems, as well as premature mortality.
Hypertension requires constant monitoring and management, and it is a condition that often goes undiagnosed until other health conditions result in an incidental diagnosis.
A good understanding of hypertension has been found to result in better management of high blood pressure. Furthermore, given that stressors related to office settings can cause fluctuations in blood pressure, most patients are advised to monitor their blood pressure at home, in relaxed surroundings.
However, healthcare providers are often overworked and unable to provide adequate information or clarify patient queries that can help in improving patient knowledge about hypertension.
The applications of artificial intelligence in medicine have been steadily increasing, and chatbots such as Bing and ChatGPT can often provide answers to patient queries. However, the accuracy and reliability of these responses need to be ascertained.
About the study
In the present study, the researchers compared two artificial intelligence tools — Bing and ChatGPT — for accuracy, reproducibility, and completeness of the responses provided to a questionnaire on methods of blood pressure control that can be performed at home.
ChatGPT was launched by Open AI in 2022 and is a tool that uses information from various sources, such as articles, websites, and books, to provide knowledge-based answers to questions from various fields, including medicine and health, in fluent and understandable formats.
The developers of ChatGPT have also optimized it to reduce irrelevant and inappropriate content, redirection, or harmful responses.
Bing bot is a similar artificial intelligence tool developed by Microsoft that responds to any question in natural human languages. This tool uses training data that enables it to provide human-like answers consistently.
Given the wide-ranging applications, the natural language, and ease of access to these tools, the researchers believe that these tools will be used by patients to obtain more information and clarity about their high blood pressure conditions and for guidance in managing and controlling hypertension at home.
The researchers used the Home Blood Pressure Monitoring (HBPM) knowledge checklist to test the accuracy, reproducibility, and completeness of the responses provided by Bing and ChatGPT.
The ten questions in this questionnaire were based on recommendations from the British Hypertension Society, the Royal College of Australia, and the American Heart Association.
The questions pertained to a wide range of subjects, such as preparation for measuring blood pressure, including rest, type of clothing, avoiding stimuli, using the blood pressure measuring cuff, and body position while taking measurements.
The questions also addressed the number of measurements to be taken and the correct method to record and read the home blood pressure monitor.
Results
The results showed that artificial intelligence tools such as ChatGPT and Bing were useful in finding answers to queries about home blood pressure monitoring. The answers provided by ChatGPT were both accurate and reliable.
However, while Bing proved to be a powerful tool for knowledge acquisition, the researchers believe that caution should be exercised while interpreting the answers provided by Bing.
While the answers to the knowledge checklist questionnaire by ChatGPT were accurate and highly reproducible, the responses provided by Bing, although highly accurate, were not consistent in their reproducibility.
Bing also had lower scores than ChatGPT in completeness. While the various responses provided by Bing had some relevant information, all the details expected for that question were not included in all the answers, indicating room for improvement in completeness.
Repeated questions were also provided with dissimilar responses by Bing, decreasing the reproducibility score.
Conclusions
Overall, the findings suggested that artificial intelligence tools or chatbots such as Bing and ChatGPT were useful in obtaining information on home hypertension management methods. However, not all tools performed at the same level.
ChatGPT responses were accurate, reproducible, and complete, but Bing, despite being a powerful tool, was lacking in reproducibility and completeness of the answers.
Therefore, while these tools are an excellent resource, there is room for improvement, and patients should interpret these results judiciously.
Understanding climate change effects on end stage kidney disease patients