Article Text
Abstract
Background Although efforts are underway to address social determinants of health (SDOH), little is known about physicians’ SDOH practices despite evidence that failing to fully elicit and respond to social needs can compromise patient safety and undermine both the quality and effectiveness of treatment. In particular, interventions designed to enhance response to social needs have not been assessed using actual practice behaviour. In this study, we evaluate the degree to which providing primary care physicians with feedback on their SDOH practice behaviours is associated with increased rates of eliciting and responding to housing and social isolation needs.
Methods Unannounced standardised patients (USPs), actors trained to consistently portray clinical scenarios, were sent, incognito, to all five primary care teams in an urban, safety-net healthcare system. Scenarios involved common primary care conditions and each included an underlying housing (eg, mould in the apartment, crowding) and social isolation issue and USPs assessed whether the physician fully elicited these needs and if so, whether or not they addressed them. The intervention consisted of providing physicians with audit/feedback reports of their SDOH practices, along with brief written educational material. A prepost comparison group design was used to evaluate the intervention; four teams received the intervention and one team served as a ‘proxy’ comparison (no intervention). Preintervention (February 2017 to December 2017) rates of screening for and response to the scripted housing and social needs were compared with intervention period (January 2018 to March 2019) rates for both intervention and comparison teams.
Results 108 visits were completed preintervention and 183 during the intervention period. Overall, social needs were not elicited half of the time and fully addressed even less frequently. Rates of identifying the housing issue increased for teams that received audit/feedback reports (46%–60%; p=0.045) and declined for the proxy comparison (61%–42%; p=0.174). Rates of responding to housing needs increased significantly for intervention teams (15%–41%; p=0.004) but not for the comparison team (21%–29%; p=0.663). Social isolation was identified more frequently postintervention (53%) compared with baseline (39%; p=0.041) among the intervention teams but remained unchanged for the comparison team (39% vs 32%; p=0.601). Full exploration of social isolation remained low for both intervention and comparison teams.
Conclusions Results suggest that physicians may not be consistently screening for or responding to social needs but that receiving feedback on those practices, along with brief targeted education, can improve rates of SDOH screening and response.
- Patient safety
- Medical education
- Healthcare quality improvement
- Graduate medical education
- Primary care
Data availability statement
No data are available. Data were collected and maintained as part of a research grant and are not available for sharing.
Statistics from Altmetric.com
Key messages
What is already known on this topic
Though efforts to address social determinants of health (SDOH) in clinical settings have already begun, minimal is known about how clinical care teams actually respond to these issues during a primary care visit.
Unannounced standardised patients, or secret shoppers, are capable of capturing patient experiences of how clinicians and support staff ask about and respond to SDOH-related concerns.
What this study adds
Cycles of audit and feedback with targeted educational information can reinforce best practices, as clinical care teams in our study became more likely to engage with and refer for their patients’ social isolation and housing insecurity, respectively, following regular cycles of audit/feedback with education.
How this study might affect research, practice, or policy
Assessment of care team responses to SDOH has the potential to impact both physician and care team training strategy and patient health outcomes.
Introduction
It is widely recognised that social determinants of health (SDOH), the conditions of the environments in which people are born, live, learn, work and play, have a substantial impact on health.1 Viewed within five general contexts (economic stability, education access and quality, neighbourhoods and the built environment, healthcare access and quality, and social and community), SDOH can both directly influence disease onset and progression and indirectly affect health through their effects on patients’ ability to access and follow through on treatment and recommendations.2 3 For example, connecting a patient with food insecurity to a meal programme can enable them to focus on adhering to medication or attend follow-up appointments rather than on their more immediate need for food. Helping a patient obtain more secure or better quality housing can decrease exposure to associated health risks (eg, toxic conditions, violence, substance use, infectious disease), decrease stress overall and therefore directly enhance health, and/or provide patients with the stability to more effectively manage their health. Social isolation may exacerbate existing depression or result from untreated depression. Care can also be adjusted to compensate for social needs (social risk-informed care), for example, by sending a patient who has problems with transportation their prescription by mail and thereby minimising the deleterious impact of that social need.4 Failure to acknowledge and/or address these social needs increases care costs,5 can compromise patient safety and is oftentimes viewed as an indicator of poor quality healthcare.6 7 Such recognition has led to many calls for action to address SDOH within healthcare settings by screening for unmet social needs and then seeking to connect patients to resources to help meet identified needs.8–12
However, screening for social determinant-based needs does not routinely occur. In one study, 42% of patients who reported not having enough resources for food, transportation and healthcare bills also stated that their clinicians only ‘sometimes or never’ were aware of these unmet social needs13 and in a large representative sample of US physicians practices, only 16% reported screening for social needs.14 Without identification, such needs cannot be adequately addressed in clinical care, leading to the risk of ineffective treatment (eg, prescribing medications the patient can’t pay for, recommending behaviour changes the patient’s environment doesn’t support, failing to incorporate air quality into the assessment of a patient’s report of breathing problems). Current studies have largely focused on screening and we could find little evidence of what physicians actually do if social needs are identified. Three specific reasons for less than ideal rates of screening and, therefore, response have been suggested: (1) Physicians’ ambivalence around their and the healthcare system’s role in addressing SDOH, (2) A lack of education about and training on how to screen for and address SDOH and (3) The complexity of doctor-patient discussions around social needs. Some physicians are ambivalent about whether screening for and responding to SDOH are essential functions they should be expected to perform,8 15 with some believing that addressing unmet social needs is beyond physician control9 or that the focus should be on structural causes and primary prevention rather than on the limited options in a single encounter.10 16–18 In addition, most physicians have not received training on how to ask about social needs and how to connect patients to necessary resources when needs are elicited. As a result, multiple physician societies, representing both the US and international coalitions, have called for providing SDOH curricula using experiential education and skills development to physicians throughout their training.18 19 There is growing evidence that such education and training, when available, can increase providers’ understanding of and intentions to address SDOH outcomes.20 21 Evidence of impact on actual practices, however, is not yet available. Finally, concerns have been raised about patients’ willingness to share SDOH22–24 and several in-depth qualitative studies of patient-physician interactions around social needs have highlighted the complexity inherent to these discussions: physicians unsure about how best to ask about social needs, patients and providers not necessarily agreeing on what counts as a social need, patients waiting to share information on social needs until they feel they can trust their provider, and difficulty in arriving at appropriate ways to address needs.25–28 These findings suggest not only the need for more education and training, but also demonstrate the methodological challenge of assessing screening and response rates when it is difficult to know whether failures to screen for or respond to social needs are attributable to the patient, the physician or their complex interaction.29
Thus, what we don’t yet fully understand is the degree to which physicians are effectively eliciting and responding to social needs and whether physicians, if provided with direct information about their effectiveness in screening for and responding to social needs, can or will improve their actual practices. We used unannounced standardised patients (USPs) as a means for controlled introduction of social needs to overcome both the methodological challenge of accurately assessing social need screening and response rates and to evaluate the impact of an intervention designed to provide feedback along with brief, targeted, practical, educational information to physicians. USPs29–34 are actors trained to present, incognito, as real patients with a standardised clinical scenario. We deployed this method as a means for delivering standardised social need information during ambulatory care visits and then assessed the response of physicians to those social needs. Given the dearth of direct evaluation of the impact of efforts designed to change physician SDOH-related practices, we designed and implemented an intervention using audit/feedback along with simple, targeted educational materials and used USP assessments of physician responses to standardised social needs to measure the impact of that intervention. Audit/feedback is a widely used strategy for assessing performance and providing feedback on that performance based on the overarching theory that healthcare professionals are motivated to improve when given feedback showing that their clinical practice is not achieving desired standards. Such approaches have been shown to be effective when sufficiently frequent, focused on increasing (rather than decreasing) behaviour, combined with education, and using trusted and relevant performance measures.35–38
Our research questions were: (1) To what degree do primary care physicians routinely elicit and respond to their patients’ social needs? and (2) Whether audit/feedback data on those baseline rates of social need screening/response, along with brief targeted education, are associated with increases in screening and response?
Methods
Study design
We implemented a preintervention and postintervention study with a proxy comparison group to evaluate the impact of audit/feedback with brief targeted education on primary care physicians' screening for and response to social needs. We took advantage of our routine system of ongoing delivery of USP visits for continuous quality improvement and physician education within primary care teams at two healthcare facilities (a large hospital and a large ambulatory care centre) in an urban, safety-net health system in New York City. For the purposes of this study, we introduced the standardised portrayal of social needs to these routine USP visits and assessed the response of the teams. Medicine resident physicians were the focus of this study and all residents on the teams were included (all five primary care teams were included in the study). There were two study phases (figure 1): the ‘pre-intervention’ period (February 2017 to December 2017) when data were collected to establish baseline rates of eliciting and responding to social needs and the ‘intervention’ period (January 2018 to March 2019) when three audit/feedback reports with data on physician responses to SDOH were shared with providers in conjunction with a series of targeted SDOH educational materials/activities. The initial design was to include a subsequent follow-up period to determine whether intervention effects, if found, were sustained. However, the healthcare system launched a new electronic medical record system in March 2019 and the associated disruption led us to cut the maintenance phase of this study.
We supplemented this pragmatic pre/post design with a ‘proxy comparison’ group in which one of the five primary care teams was randomly selected to serve as the comparison team and did not participate in the intervention. This design does not represent a controlled design but offers at least a partial window into the counterfactual during the intervention period—potentially capturing the effects of increased emphasis on SDOH within both the local and national healthcare environments separated from the intervention effects, although with all the limitations associated with a ‘proxy comparison’ versus a randomly assigned control group.
Delivery of controlled SDOH: USP cases
We routinely deliver six unique USP case scenarios designed to represent the common primary care conditions and be representative of the patients in our healthcare system as part of our residency training and quality improvement programme. For this study, we programmed specific underlying social-determinant related needs into each of the cases (table 1). We focused on housing and social needs and chose specific issues within those categories in order to target needs that were closely linked to health and that could be addressed in a primary care visit, either by adjusting care plans or connecting to services. Actors were trained to divulge their specific housing and social issues only if asked—physicians had to screen for these social needs. They were instructed to broadly share their concerns if physicians asked them anything about their housing and social/family situations (eg, sharing that their apartment was old/dilapidated or that they were pretty ‘crowded in’ when asked about housing) but to only provide full information (eg, mould in the apartment they thought might be related to their asthma or that the crowding involved several generations together that made the patient concerned about infection transmission) if the physician provided them opportunities to do so by asking for more information or following up on and exploring the initial response. USPs completed a comprehensive behaviourally anchored checklist after each visit capturing physician response to their social needs.
USPs were trained in consistent and realistic case portrayal and standardised assessment during 4 hours of inperson character rehearsal and 2 hours of checklist training. Standardisation of case portrayal and presentation during the clinical encounter and quality of assessment were ensured through monitoring of visit audiotapes and chart reviews. Detection rate, as measured through routine surveying of participants as part of our USP quality improvement system, remained low throughout the study period (~10% of visits, and almost always after the visit).
The intervention: audit/feedback reports and SDOH educational materials
The intervention consisted primarily of three team-based audit/feedback reports: the first marked the start of the intervention period and included data on physician screening and response rates from throughout the baseline period and the second and third were delivered 5 months and 9 months later (see figure 1 for timeline). Reports depicted, for the specified team and overall, the per cent of USP visits during the above defined periods in which the physician fully elicited and then responded to scripted social needs for that team. Individual physician-level data were not provided as USP visits were delivered to teams and with as many as 20 physicians/team all with varied schedules and patient panel sizes, no single physician saw a sufficient number of USPs. The first audit/feedback report covered the baseline period and included about 20 USP visits/team over 11 months (2–3 visits/month per team), the second audit/feedback report covered months 12–16 (60 visits, 10–14 visits/team) and the third audit/feedback report covered months 17–20 (59 visits, 9–13 visits/team). Each report also included a short educational component that was designed with input from the health system’s social work and community services referral teams and was intended to developmentally introduce and reinforce clinical guidelines for addressing SDOH. The initial baseline report included a half-page summary that defined SDOH and provided referral options, including points of contact within the social work team. The second report detailed instructions on documentation of social needs in the electronic health record (EHR) through billing codes (which can be used to track SDOH in administrative data sets) and provided updated information on SDOH referral processes. The third report provided information on the kinds of resources patients were frequently linked with in order to demonstrate the practical utility of making SDOH referrals. Reports were distributed via email and also at team and residency programme meetings, and included short verbal reviews of the data and the educational materials, and the latter were also posted in the intervention teams’ staff conference rooms. During inperson sessions, team members could engage in relatively brief discussions about how to improve screening or response. Multiple modes of dissemination were used to address the practical constraints of busy, safety-net clinics serving vulnerable populations, including staffing schedules and turnover, competing demands for attention, and the importance of bringing the reports and education to the teams rather than pulling them out of the clinic to attend a separate event. Feedback data and educational information were spaced out over three sessions to build on the evidence base for audit/feedback and continuous improvement models and to take advantage of spaced repetition to reinforce learning and skills development. An example of an audit/feedback report is available as a online supplemental file.
Supplemental material
Measures
Postvisit checklists were completed by the USPs and included assessments of the clinic, the team, and the physician’s core clinical skills, and for this study, an in-depth assessment of the physician’s SDOH practices. The screening section of the checklist included questions about whether the physician elicited their housing and social concerns broadly, and if so, was the USP able to share their specific concern (eg, ‘I’m concerned that the mould in my bathroom might be making my asthma worse’ or ‘I’m feeling really isolated since my breakup’). If the social needs were fully elicited, USPs recorded physicians’ responses to those concerns in the following categories ordered from worst to best practice: Responded negatively (was judgmental/dismissive), Did not respond at all (ignored my statement), Acknowledged my social need, Explored my social need, Responded to my social need by giving advice, or Responded to my need by offering resources of any kind. We considered appropriate responses to the housing situation to be any offer of resources (eg, contact information for a housing advocacy organisation or legal services) or direct referral (eg, to social work) and for the social concern, given that direct services to combat social connectedness are not readily available, fully acknowledging and exploring the patient’s social concern. These assessments represent our outcome variables: the per cent of visits in which the housing social need was fully elicited and then, in those visits where elicited, the per cent of visits where the physician actually provided resources/referral; and the per cent of visits in which the patient’s social isolation was elicited, and then when elicited, the per cent of visits where the physician fully explored the patient’s social situation.
Human subjects and ethics considerations
this study met our institutional review board’s criteria for self-certification as a programme evaluation/quality improvement project and was not considered to be human subjects research as the activities were part of ongoing quality improvement and the research team had access only to fully de-identified data. physicians and primary care teams were notified in advance that USPs would be sent to their clinics as part of ongoing quality monitoring/improvement and education/training efforts.
Analyses
Rates of screening for and responding to identified social need for all visits for each SDOH concern (housing and social) between the two periods (pre/baseline vs intervention/post) were compared by χ2 tests. These comparisons were conducted for combined visits across all intervention teams (n=4) and then separately for the proxy comparison team. We were limited in the number of USP visits that could be delivered to each team to avoid overuse of resources and minimise detection (about 2–3/team each month), and therefore were not able to examine more granular effects (eg, month to month changes). Data on the distribution of USP visits delivered were first compared between the baseline and intervention periods to ensure we achieved our intended goal of rough equivalence between these two time periods in terms of type of case, team, and programme and year of the resident physicians.
Results
Delivery of visits
A total of 108 USP visits were completed during the baseline period (February 2017 to December 2017), 85 to the intervention teams and 23 to the proxy comparison team and a total of 183 visits were completed during the intervention period (January 2018 to March 2019), 150 to the intervention teams and 33 to the proxy comparison team. Visits were implemented continuously at a similar rate (2–3 visits/team per month) across the entire study period. A total of 93 resident physicians were distributed across these teams and data collection periods.
Comparability of baseline and intervention time periods
Prior to analysing core outcomes, we compared visit characteristics between the baseline and intervention periods to assess the distribution of possible confounding variables (table 2). There were no statistically significant differences in the distribution of USP visits in terms of case types, teams/sites or resident physician postgraduate year (PGY) level and programme (Internal Medicine Categorical vs Primary Care) between the baseline and intervention periods for the intervention teams. However, the proxy comparison team did have significantly more visits seen by a PGY2 resident during the baseline period (16 of 23, 70%) than in the intervention period (9 of 33, 29%) (χ2=11.30, p=0.004). However, we did not find any significant differences in the rates of screening for or responding to SDOH by PGY.
Outcomes
Figures 2 and 3 display the core results. At baseline, the housing need was elicited in about half of the visits (39 of 84, 46% (n=39)) for the intervention and 61% (14 of 23, n=14) for the comparison group. The rate of eliciting housing issues increased significantly from baseline (46%, 39 of 84 visits) to intervention (60%, 90 of 150 visits) periods for the intervention teams while we found a decline in the proxy comparison team (from 61% (14 of 23) during the baseline to 42% (14 of 33) during the intervention period). In the visits where the housing need was elicited, the baseline rates of response (providing resources or direct referral) were low for both intervention and comparison teams (15%, 6 of 39 and 21%, 3 of 14, respectively) but in the intervention teams, that rate increased to 41% (37 of 90) during the intervention period, while the proxy comparison team’s rate did not significantly increase (29%, 4 of 14)).
At baseline, the intervention teams and the proxy comparison team had the same rate of eliciting the social isolation concern (39% of visits, 33 of 85 and 9 of 23, respectively). By the intervention period, however, the teams that received the audit/feedback reports and SDOH educational materials elicited the social isolation need in more of the visits (53%, 78 of 148), while the proxy comparison team elicited the social isolation concern in slightly (but not significantly) fewer visits (32%, 10 of 31). In terms of responding to the patients’ social isolation, rates of fully acknowledging and exploring did not change significantly between the baseline and intervention periods for either the teams that received audit/feedback reports or the comparison team.
Discussion
In summary, we examined the impact of audit/feedback of USP visits on resident physicians’ response to social needs and found that the rates of both screening effectively for and responding to housing concerns improved significantly for the intervention teams but not in the proxy comparison team and that rates of eliciting social isolation concerns also improved significantly from baseline to intervention periods in the intervention teams but not for the comparison team. Rates of responding to social isolation did not change. If our proxy comparison group provides a reasonable internal validity check, ‘controlling’ for secular trends with respect to broader healthcare system and national efforts to increase attention to SDOH in clinical care, then we can have some confidence that our use of USPs to assess and then provide feedback on how resident physicians address social needs, combined with targeted educational materials, delivered via the reports and reinforced inperson when possible during existing team huddles/meetings, led to increases in the rates of eliciting both housing and social isolation-related needs and to effectively responding to housing insecurity needs.
We were not surprised to see that rates of fully exploring social isolation did not change from baseline to intervention given that there is, currently, a lack of consensus on and insufficient evidence for how best to intervene in addressing loneliness and social isolation.39 What we believe is worth noting, however, is that even with improvement, resident physicians far too often failed to adequately ask about their patient’s housing situation (the USP did not have an opportunity to reveal an important housing issue in 40% of visits) or their social situation (47% of visits) and once elicited, failed to take minimum action to address the issue more than half the time. That a key social need was not elicited in over half of visits in a healthcare system serving vulnerable patients, even after targeted audit/feedback, during a time when the healthcare system was encouraging the use of SDOH screening forms and processes and physician societies and the literature were persistent in their calls for addressing SDOH, suggests we must devote more resources to education, training and supports to ensure our physicians elicit (and act on) this essential patient information, especially knowing that most ‘real’ patients may not volunteer their social needs issues as easily as our USPs were trained to do. While the patients in this study were actors, failure to find out about the mould in an actual asthma patient’s apartment could lead to escalating unnecessary asthma treatment for a ‘real’ patient, a stark illustration of how addressing SDOH is essential to patient safety and quality care.
We believe that the following features of the intervention, included in the design based on the literature on audit/feedback,35–38 may have contributed to the improvements in response to SDOH: (1) The fact that evidence from USP visits of failing to screen for and respond to social needs was difficult to refute; (2) That the specific cases and associated educational materials emphasised the consequences for the patients’ care of failing to address the social need, and/or (3) That finding out that some of their fellow resident physicians were routinely making referrals to social work for housing issues and/or directly providing resources may have been motivating. However, our study was not designed to provide evidence for the specific components or levers of change within the intervention. Further research is needed to determine which aspects matter the most for enhancing physicians’ SDOH practices and how best to scale up education and intervention efforts that incorporate those aspects and deliver them more broadly throughout physician training. Healthcare systems have increasingly built screening for social needs into their EHRs12 and while building this process into the structured workflow is an important step in the right direction, our results highlight the need for physicians to be effective in not just asking about but also fully exploring and eliciting patients’ social needs and then addressing those identified needs. And while many view USP programmes as difficult and costly to implement, we believe and this study supports that this methodology can be seamlessly integrated into primary care and provide ongoing continuous feedback on targeted outcomes, offer flexibility to introduce specific scenarios, and can serve as a vehicle for integrating training and education directly into the operations of a busy, safety-net hospital system.
Despite the potential power of USP data to support quality improvement efforts, the ability to introduce controlled ‘stimuli’ into the noisiness of healthcare, and the unique insight these visits can provide on the details and specifics of care, USP visits do carry with them many research limitations. USP visits are most easily delivered as new visits and the resident may have decided to simply document the social determinants on the first visit while planning to address them in a subsequent visit. However, in additional analyses, reported elsewhere we found that rates and quality of documentation were low across visits (<30% for housing insecurity, <10% for social isolation).40 We are unable to deliver more than an average of two to three visits/week to each clinic—more than that is likely to lead to detection, overwhelm the clinic and siphon needed resources from actual patient care. With visits as the unit of analysis, this constrains sample sizes and while an interrupted time-series analysis of say, monthly rates of SDOH response would have been more ideal, we were not able to deliver enough visits per month to support those analyses. Questions of the authenticity of patient portrayal should always be raised in studies using USPs, though we devoted substantial time and resources to training and quality assurance to have confidence in the ‘real-ness’ of portrayal.
The use of a ‘proxy’ comparison is a limitation of our study that goes beyond the USP method. The intervention was not delivered through true random assignment and the comparison consisted of one team with a small sample size of visits and therefore is subject to influence by all the outlier effects, selection biases, situational events and contextual factors associated with a single team and a small n. Finally, our study involved internal medicine residents training in a large, urban, safety-net hospital serving a vulnerable patient population and the results may not generalise to other samples and settings.
Beyond the inherent limitations of the USP methodology, we believe it ultimately offers a cost-effective means for supporting continuous quality improvement within a healthcare system and, in particular, that it can be effectively deployed to systematically ‘test’ and provide feedback on how primary care teams address SDOH. Alternative methods for providing ongoing audit/feedback can be used, such as patient surveys and chart reviews, but these lack the control and targeting of USPs. Without this control, it is difficult to determine whether low rates of identification of SDOH are due to low rates in the population, patients’ hesitancy to divulge needs, and/or physicians’ failure to even ask at all about or to fully elicit social needs. One clear advantage of the USP method is that it allowed us to accurately identify when patients’ social needs were missed or not addressed—failures that cannot be attributed to, for example, the patient not wanting to share that information. It’s these missed opportunities that provide the ultimate rationale for investing in efforts to integrate SDOH into primary care: in our two cases, actually addressing the patients’ housing and social isolation issues could lead to significant downstream savings to the healthcare system and substantially improved health outcomes for the patients themselves.
Conclusion
Overall, these results provide important evidence that audit/feedback combined with targeted education can help improve resident physicians’ response to SDOH in primary care practice.
Data availability statement
No data are available. Data were collected and maintained as part of a research grant and are not available for sharing.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
Acknowledgments
The authors thank Health and Hospital leadership for their continued passion for the use of USPs in building a learning health system. The authors also thank the standardised patients, faculty, ROMEO group and learners for creating a supportive training environment.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors have contributed substantially to this work and to the development of this manuscript. CG serves as guarantor.
Funding This work was supported by the Agency for Healthcare Research and Quality (AHRQ 1R18HS024669-01) and the Health Resources & Services Administration (HRSA 15-A0-00-004497).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.