Article Text
PDF
Abstract
Direct-to-onsumer telemedicine (DTCT) has become popular as an alternative to traditional care. However, uncertainties about the potential risks associated with the lack of comprehensive quality evaluation could influence its long-term development. This study aimed to assess the quality of care provided by DTCT platforms in China using unannounced standardised patients (USP) between July 2021 and January 2022. The study assessed consultation services on both hospital and enterprise-sponsored platforms using the Institute of Medicine quality framework. It employed 10 USP cases, covering conditions such as diabetes, asthma, common cold, gastritis, angina, low back pain, child diarrhoea, child dermatitis, stress urinary incontinence and postpartum depression. Descriptive and regression analyses were employed to examine platform characteristics and compare quality across platform types. The results showed that of 170 USP visits across 107 different telemedicine platforms, enterprise-sponsored platforms achieved a 100% success in access, while hospital-sponsored platforms had a success rate of only 47.5% (56/118). Analysis highlighted a low overall correct diagnosis rate of 45% and inadequate adherence to clinical guidelines across all platforms. Notably, enterprise-sponsored platforms outperformed in accessibility, response time and case management compared with hospital-sponsored platforms. This study highlights the suboptimal quality of DTCT platforms in China, particularly for hospital-sponsored platforms. To further enhance DTCT services, future studies should compare DTCT and in-person care, aiming to identify gaps and potential risks associated with using DTCT as alternatives or supplements to traditional care. The potential of future development in enhancing DTCT services may involve exploring the integration of hospital resources with the technology and market capabilities of enterprise-sponsored platforms.
- Health services research
- Quality measurement
- Performance measures
- Patient-centred care
Statistics from Altmetric.com
Introduction
Direct-to-consumer telemedicine (DTCT) signifies a revolutionary global healthcare delivery model. It involves patients independently initiating medical services remotely, engaging directly with healthcare providers through text messaging or video/phone calls. By bypassing traditional intermediaries like referral clinicians or facilitators, DTCT empowers patients to access medical care swiftly and efficiently.1 In China, DTCT is burgeoning, boasting over 1700 registered platforms,2 each serving as an individual website or application for DTCT services. These platforms are categorised into two main types: hospital sponsored and enterprise sponsored. Hospital-sponsored platforms, associated with single physical hospitals, primarily use in-house medical staff and offer streamlined functions due to resource limitations. In contrast, enterprise-sponsored platforms, supported by larger corporations, provide access to a wider network of licensed physicians and offer a diverse range of functions with sophisticated user interfaces.3 Despite enhanced accessibility and convenience compared with in-person care, DTCT faces quality challenges, such as communication difficulties and antibiotic misuse.4 5 However, research on DTCT quality remains limited, especially in the context of China. To address this gap, our study employs unannounced standardised patients (USPs)—individuals trained and validated to portray specific medical conditions in a consistent and standardised manner6 7—to assess DTCT quality in China across different platform types based on the Institute of Medicine (IOM) quality framework.
Method
This cross-sectional study examined both types of DTCT platforms that offered Chinese language services. The study was conducted between July 2021 and January 2022. Prior informed consent was waived due to minimal risk, and all analyses were performed on fully deidentified aggregated data.8
To assess consultation service quality, we employed 10 different USP cases, each representing a specific medical condition (details in online supplemental eMethods and online supplemental example of a USP case). Following thorough training and assessment, we selected 15 qualified USPs from the initial pool of 25 candidates. Each case was assigned at least one USP. As USPs’ initial requests for physician consultations could be denied for various reasons, mirroring real-world scenarios, each case made consultation requests until they completed at least five consultations on each platform type. The USPs captured screenshots and recorded the consultation process, including any failed attempts and reasons for failure. We evaluated the access success rate and assessed consultation quality using the IOM quality framework9 (table 1).
Supplemental material
Quality outcome indicators
We conducted descriptive and regression analyses to evaluate platform characteristics and quality outcomes. Regression analyses compared quality differences using ordinary least squares and logistic regressions, with hospital-sponsored platforms as the benchmark and controlling for patient case fixed effects. Adjusted differences and 95% CIs were reported, adjusted for platform-level clustering. Statistical significance was set at α=0.05. Stata SE (V.16.0) was used for all analyses.
Results
Our study involved 170 visits, with 52 visits on 10 enterprise-sponsored platforms and 118 visits on 97 hospital-sponsored platforms (summarised in online supplemental eTable 1). The overall access rate was 63.5% (108/170). Enterprise-sponsored platforms achieved a 100% access rate (52/52), significantly higher than hospital-sponsored ones (56/118, 47.5%) (p<0.001). Common reasons for unsuccessful consultations included incomplete functions, like platforms claiming to offer DTCT services but lacking an accessible feature for initiating online consultations with physicians (25/62, 40.3%), and no response (13/62, 21.0%).
Of 108 successful consultations, 49 consultations (45%) received a correct diagnosis, while adherence to published guidelines was low for consultation (15%) and management decisions (31%). On average, physicians took 3 hours and 47 min to respond, with the total interaction time spanning 12 hours and 19 min. After controlling for disease case fixed effects and adjusting SEs for clustering at platform level, enterprise-sponsored platforms had higher rates of completed management decisions, shorter response times and higher costs (table 2).
Comparison on main quality outcomes between platform types for successful consultations
Discussion
In this study, we rigorously evaluated the quality of care across two types of DTCT platforms in China using USPs. With 108 successful visits out of 170 attempts, we found a diagnostic accuracy of approximately 45%, along with a decline in completion rates for recommended management decisions to 31%. This may be linked to the limited inquiries during USP encounters, with clinicians asking only about 15% of recommended consultation questions per visit. Besides traditional quality metrics, the timeliness of DTCT services is concerning, with a response time of 3 hours and 47 min and an overall interaction time of 12 hours and 19 min. In China, DTCT predominantly operates asynchronously, leading to these extended durations. Despite offering a more flexible alternative to in-person counselling by eliminating the need for travel, prolonged waiting times may still impact user satisfaction and the perceived effectiveness of DTCT platforms. Future research should prioritise enhancing the timeliness of DTCT services to ensure prompt access and timely interactions. The patient-centredness score averaging 2.4, indicating a medium to low level of patient-centred care,10 is potentially influenced by less satisfactory outcomes discussed earlier in terms of effectiveness, safety and timeliness. These findings raise concerns about how effective DTCT services are. Further evaluation, including a direct comparison with in-person care, is needed for a clearer understanding of their quality. This can guide improvement measures, especially when DTCT services act as alternatives or supplements to traditional in-person care.
Notably, enterprise-sponsored platforms achieved 100% access success, surpassing hospital-sponsored ones at 47.5%. They exhibited superior performance in response times and completion rates for management decisions. Despite recent growth and policy support,3 hospital-sponsored platforms seem to be in early developmental stages, potentially limiting medical resource accessibility. These findings challenge a marketing survey suggesting a preference for hospital-sponsored platforms,11 emphasising higher access denial risks and less timely responses for consumers on these platforms.
This study was limited by the use of a uniform USP for each case. Using standardised scenarios with different USPs potentially allows for a comprehensive assessment of equity. However, due to constraints imposed by scripted scenarios in our study, this aspect was not explored.
Our study highlights the suboptimal quality of DTCT in China, specifically disparities between hospital-sponsored and enterprise-sponsored platforms. These findings likely echo broader challenges and principles inherent in DTCT globally. As DTCT gains momentum after COVID-19, future research becomes critical to effectively address these issues.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and was approved by Xiangya School of Public Health (IRB No XYGW-2021-37). Prior informed consent was waived due to minimal risk and no individually identifiable information on physicians.
Acknowledgments
We sincerely thank Xiaohui Wang, Yaolong Chen, Yun Lu, Xiaojing Fan, Zhongliang Zhou, Jay Pan, and Chengxiang Tang for their unwavering leadership in development and management the SP cases. We also apperciate Lu Liu, Chunping Li, and Huanyu Hu for their their diligent efforts as project assistants, as well as all the standardized patients and study coordinators for their hard work.
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors WG conceived the study. ZZ coordinated the daily implementation of this study under the supervision of DRX, YC and WG. ZZ carried out data analysis and composed the initial manuscript draft, receiving guidance from WG and DRX. All authors contributed to critical review of the manuscript and approved the final draft.
Funding This study was funded by China Medical Board (20-368), Swiss Agency for Development and Cooperation (81067392) and the National Natural Science Foundation of China (82273643).
Competing interests None declared.
Provenance and peer review Not commissioned; internally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.