So you want to quit smoking. But you want to do it right, with expert advice and evidence-based information. Should you ask Siri?
This week my co-author Nick Wilson and I published results of a pilot study reporting how effective personal digital assistants are at providing information or advice to help you quit smoking.
As far as we are aware our study is the first study looking at whether Siri or Google Assistant can help you quit.
The internet is widely used for obtaining health-related information and advice. For example, in the United Kingdom, 41% of internet users report going online to find information for health-related issues, with about half of these (22% of all users) having done so in the previous week.
We compared voice-activated internet searches by smartphone (two digital assistants) with laptop ones for information and advice related to smoking cessation.
We asked Siri and Google Assistant three sets of questions. We entered the same questions into Google as an internet search on laptops.
The first set of questions were adapted from the ‘frequently asked questions’ on the UK National Health Service (NHS) smokefree website.
The next set of questions were related to short videos on smoking-related disease produced by the Centers for Disease Control and Prevention (CDC) in the USA.
The final set of questions we devised to test responses to a range of features such as, finding smoking-related pictures, diagrams, instructional videos; and navigating to the nearest service/retailer for quitting-related products.
We graded the quality of the information and advice using a three tier system (A,B,C) where A represented health agencies which had significant medical expertise, B was for sites with some expertise (e.g. Wikipedia) and C was for news items, or magazine style content.
Google laptop internet searches gave the best quality smoking cessation advice 83% of the time, with Google Assistant on 76% and Siri 28% (equal firsts were possible).
The best search results by any device used expert (grade ‘A’) sources 59% of the time. Using all three methods failed to find relevant information 8% of the time, with Siri failing 53% of the time.
We found that Siri was poor when videos were requested according to the content the might contain, all three tools sometimes returned magazine or blog content instead of professional health advice, and we found that all tools had trouble when gay and lesbian-specific information was requested.
A weakness of our small pilot study was that we only considered the first result returned in each search.
Overall, while expert content was returned over half the time, there is clearly room for improvement in how these software systems deliver smoking cessation advice. We would encourage software firms to work with professional health organisations on ways to enhance the quality of smoking cessation advice returned.
See Adapt Research Ltd’s related blog: ‘To vape or not to vape… is not the right question’
Adapt Research Ltd 