Article Text
PDF
Abstract
Background Audit and feedback (A&F) interventions improve patient care but may result in unintended consequences. To evaluate plausible harms and maximise benefits, theorisation using logic models can be useful. We aimed to explore the adverse effects of colonoscopy A&F using a feedback intervention theory (FIT) dark logic model before the National Endoscopy Database Automated Performance Reports to Improve Quality Outcomes Trial study.
Methods We undertook a qualitative study exploring A&F practices in colonoscopy. Interviews were undertaken with endoscopists from six English National Health Service endoscopy centres, purposively sampled for professional background and experience. A thematic framework analysis was performed, mapping paradoxical effects and harms using FIT and the theory of planned behaviour.
Results Data saturation was achieved on the 19th participant, with participants from nursing, surgical and medical backgrounds and a median of 7 years’ experience.
When performance was below aspirational targets participants were falsely reassured by social comparisons. Participants described confidence as a requirement for colonoscopy. Negative feedback without a plan to improve risked reducing confidence and impeding performance (cognitive interference). Unmet targets increased anxiety and prompted participants to question messages’ motives and consider gaming.
Participants described inaccurate documentation of subjective measures, including patient comfort, to achieve targets perceived as important. Participants described causing harm from persevering to complete procedures despite patient discomfort and removing insignificant polyps to improve detection rates without benefiting the patient.
Conclusion Our dark logic model highlighted that A&F interventions may create both desired and adverse effects. Without a priori theorisation evaluations may disregard potential harms. In colonoscopy, improved patient experience measures may reduce harm. To address cognitive interference the motivation of feedback to support improvement should always be clear, with plans targeting specific behaviours and offering face-to-face support for confidence.
Trial registration number ISRCTN11126923.
- audit and feedback
- healthcare quality improvement
- qualitative research
Data availability statement
Data are available upon reasonable request. All participants provided written consent for non-identifiable publication of transcript extracts and direct quotations from data. Data were accessed in conjunction with Newcastle University data security policy. The data sets generated and/or analysed during the current study are not publicly available due to possible identification of participants through triangulation but are available from the corresponding author upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Background
Audit and feedback (A&F) interventions have been shown to improve compliance with desired practice in healthcare professionals; however, effect sizes have varied and some interventions are not successful.1 There are calls for the more explicit use of theory to understand mechanisms of change in behaviour change interventions (BCIs) and to develop logic models to inform implementation.2 3 Theory has explained some variability4 5; however, models often only focus on the intended benefits of the intervention and an additional explanation for variable effect size are unintended effects. In the public health sector BCIs involving human agency and complex social systems have been demonstrated to potentially have unintended or harmful consequences.6 These have included BCIs associated with higher rates of adolescent problem behaviour,7 teenage pregnancy8 and rates of sexually transmitted infections.9 It is hypothesised that modelling underlying mechanisms of paradoxical effects and harms provides an opportunity to avoid or minimise these problems.10
‘Dark logic’ is defined as the mechanisms by which an intervention hypothetically has adverse effects on the outcomes of interest (‘paradoxical effects’) and other outcomes (‘harmful externalities’). Dark logic models are developed by scrutinising models of intended change and their assumptions using a priori theorisation to actively hypothesise paradoxical effects and harms. These are recommended in public health and prevention science interventions,11 12 and have been used to critique and analyse public health intervention policy in the COVID-19 pandemic.13 14 To the knowledge of the authors, they have not been used in the development healthcare A&F BCIs.
Colonoscopy is a medical procedure that involves an endoscopist inserting a camera into the large bowel (intubation) then withdrawing the camera looking for pathology (withdrawal); beforehand, the bowel is cleansed (bowel preparation) to allow visualisation.15 Colorectal cancer (CRC) arises from polyps, and polyp detection and resection at colonoscopy is pivotal in preventing CRC. Performing colonoscopy can be challenging and poor-quality colonoscopy has serious consequences. Endoscopists with lower polyp detection rates have higher rates of CRC after colonoscopy.16–18
The UK government has supported the implementation of a quality improvement programme in endoscopy, overseen by the Joint Advisory Group (JAG) on endoscopy and the British Society of Gastroenterology, and associated with a reduction in CRC mortality.19 The programme introduced a bowel cancer screening programme (BCSP) with advanced accreditation of screening endoscopists and national key performance indicators (KPI) of colonoscopy quality, including completeness of procedure (caecal intubation rate), polyp detection rates, withdrawal time and comfort.20 21 Previous trials of A&F to improve colonoscopy performance have had heterogenous results; this is hypothesised to be due to colonoscopy being a complex motor skill and poor implementation of BCIs.22
In the development of the National Endoscopy Database Automated Performance Reports to Improve Quality Outcomes Trial (NED-APRIQOT), we undertook a wider qualitative interview study to explore the phenomenon of current A&F practices in colonoscopy to develop a BCI prior to its implementation later in 2020.23 Feedback intervention theory (FIT)24 has been demonstrated to be a suitable theoretical model for change in A&F interventions in healthcare settings,5 and has been recommended for the development of A&F processes in endoscopy.22 This FIT model was used to explore paradoxical effects, whereby a BCI increases a behaviour it seeks to prevent, and harmful effects to patient care. These are harms which differentially affect patients in the care of practitioners who, themselves, are the target of BCIs. Changes in intention partly predict behaviours25 and the theory of planned behaviour (TPB) is moderately effective at predicting intention and behaviour.26 The Cochrane review of A&F suggested that TPB is particularly useful to explore normative comparisons and was used to map participants’ beliefs within FIT.27
The aim of this paper is to describe the phenomena of potential harms and adverse outcomes in A&F processes in endoscopy arising at interview using a theoretical model based on FIT, to inform the design of a future BCI.
Method
Independent endoscopists were recruited for face-to-face audio-recorded semistructured qualitative interviews at their workplace; these were followed by cognitive interviews assessing a draft BCI, reported in a separate study.28 Interviews lasted up to 60 min. Clinical leads of English National Health Service endoscopy centres eligible for the NED-APRIQOT study in the Northern region or West Midlands were contacted by email. Sites which responded were selected with convenience sampling for participants’ availability. Eligible endoscopists23 were purposively sampled with criteria comprising length of endoscopy experience and professional role (clinical lead, clinical nurse endoscopist, gastroenterologist, surgeon and trainee) or to aid data saturation. Up to five endoscopists were recruited at each site; recruitment continued until sampling strata were filled and data saturation was reached, defined as no new themes arising in the last three interviews after 10 interviews.29
Interviewees were provided with a participant information sheet, explaining interviews would cover behaviours in endoscopy and A&F, and gave written consent. A topic guide was used, reviewed and revised (if needed) after each centre’s interviews to facilitate depth and data saturation (online supplemental appendix 1). Interviews were transcribed removing any identifiable information for analysis with demographic data pseudoanonymised using a unique participant identifier, and the interviewer (JC) kept a reflective log. Participants were provided with a copy of their transcripts to ensure anonymity and accuracy, and to drive meaningful conclusions from extracted quotes.
Supplemental material
A framework method analysis was undertaken; FIT informed an initial logic model for the intended effects of A&F intervention in endoscopy, providing variables of interest, and a preliminary basis for a relationship between codes in the analytical framework FIT describes behaviours as tasks at three possible levels:
Meta-task–beliefs about the self that are required to perform a task.
Task motivation—where one applies an already learnt behaviour.
Task learning—new tasks where one focuses on motor movements.
FIT suggests that if an individual identifies a gap between current performance and a target, they may adopt a strategy to reduce this gap and develop a coping mechanism, abandon the standard, reject the feedback message or change the standard (figure 1).
A logic model of audit and feedback processes based on feedback intervention theory.
The TPB30 was used to explore beliefs about behaviours within FIT. The TPB identifies beliefs as:
Behavioural beliefs—attitudes towards and effects of behaviours.
Control beliefs—perceived control of behaviours by participants.
Normative beliefs—perceived social pressures around behaviours.
The framework method analysis used inductive ‘open coding’, based on Gale et al31 involving the following steps:
Preliminary reading of full transcripts, ensuring accuracy and adding context.
Generation of initial descriptive codes, inductive ‘open coding’ paraphrasing the text ideally using participants’ own words.
Developing an analytical framework after eight transcripts, grouping codes into subthemes tagged to FIT domains or TPB beliefs. Codes which did not sit within FIT or TPB were analysed in ‘bucket’ subthemes.32
Applying the analytical framework indexing subsequent transcript codes.
Charting data: subthemes and their relationships were reviewed and mapped to FIT themes and a FIT logic model for behaviour change with corresponding quotes from data to ensure accuracy.
The dark logic model was developed through scrutinising the model for paradoxical effects and harms. Keeping in line with focus of this paper, themes of intended benefits and wider A&F phenomena are not reported. Interviews and analysis were undertaken by a single researcher (JC), codes were logged with an audit trail. Themes were reviewed with original quotation data to ensure accuracy and triangulated with observation data and personal reflection from the time of interview. As the coding and analysis progressed, the authors met to critically review, challenge and discuss findings. Data are reported in two major areas: paradoxical effects and harmful effects. Illustrative quotes are provided.
Results
Six endoscopy centres were recruited from April 2019 to January 2020, four in the North East of England and two in the West Midlands. Centres had a median of seven endoscopy rooms across sites; the range (two to eight rooms) demonstrated a good range of small to large endoscopy centres.
Saturation of themes was achieved by participant 19. Ten of the 19 participants identified as being female. Sampling criteria for professional background and experience were fulfilled and shown in table 1; there was a median of 7 years’ endoscopy experience, with a range of 2–29 years.
Endoscopy centres and their participants’ roles
All sites provided endoscopists with A&F data by email at least every 6 months, as part of routine practice in line with JAG recommendations. This provided a comparison of performance to national standards and a social comparison to others in the endoscopy centre. Persistent underperformance was managed by centre leads as recommended in national guidance.20 33
Paradoxical effects
Paradoxical effects are summarised in figure 2 in orange.
Dark logic model for audit and feedback harms in endoscopy using feedback intervention theory.
Rejecting the gap: seeing peers’ performance
All centres provided national standards for minimum performance,20 and a normative comparison to peers with information about other local endoscopists’ performance. Participants saw the aspirational performance of comparable peers as motivating and benchmarked against colleagues whom they recognised as experts. ‘I looked at who had the best polyp detection rate and I thought, “I would like my polyp detection rate to be nearer to that”’ (Participant (P) 13).
Participants often identified themselves in a social professional group of peers with comparable case mix, job plan or professional background, and this identity was viewed as important to them. Within these groups, social norms were identified for performance and endoscopists had a perception of ‘what your level is at’ (P9). Performance being perceived as similar to others within this referent group reduced motivation to improve, even if this performance was below a centre-wide average or an aspirational target achieved by other peers. ‘One of our other nurse endoscopist colleagues who was always the same as me [below target] and we did a lot of endoscopy, us two … the ones who actually had higher detection rates than me were actually, I thought personally, not as good endoscopists’ (P10, table 2: subthemes (S) 1–2).
Paradoxical effect subthemes and illustrative quotations
Cognitive interference and quitting
Participants described that thinking about A&F data could impede performance and risked endoscopists getting ‘bogged down’ (P10). One participant described ‘thinking about your figures, it’s probably not brilliant …I mean I find it can be really emotionally draining’ (P1). Participants described how negative feedback, without a plan to improve performance, reduced their confidence, worsened performance and increased the risk of quitting colonoscopy (table 2: S3).
Colonoscopy is a complex motor skill, with many underlying task motivation processes which participants were able to describe, including: position changes, managing air and resolving loops (table 2: S4). However, some of these task motivation processes were higher level behaviours which participants struggled to describe: one participant described ‘using “the force”’ (P19, table 2: S4), suggesting colonoscopy is related to confidence, a meta-task self-perception behaviour. A clinical lead described the reason endoscopists quit endoscopy ‘wasn’t because of any technical ability, it was just [their] confidence’ (P1). There were concerns that an individual’s feedback ‘might end up being a …bit destructive’ with the risk that ‘you think I’m not very good here… that might be quite demotivating’ (P1). However, participants recognised being receptive to negative feedback was an important part of a clinical role and a culture for quality improvement.
Participants generally accepted that KPIs showing repeated underperformance suggested something was wrong, and if an endoscopist was unable to address underperformance they should consider or may be asked to stop scoping: ‘if we’re not good at something and we’ve tried to address it and we can’t find what’s wrong and you can’t address it, then maybe you just need to think about something else or giving it up’ (P18). Stopping scoping was described as having financial and psychological consequences for endoscopists, ‘If [the A&F process] wasn’t successful … we couldn’t really maintain her as an endoscopist and therefore there would be a big salary hit… But I also think it was more she regarded herself as a failure’ (P1).
These high stakes lead to cognitive interference and anxieties focused on possible motives of the feedback other than to improve performance. These suspected ulterior motives included accusing endoscopists of not doing enough, persecuting endoscopists who were wrongly perceived to be incompetent and policing performance to stop people from scoping (table 2: S5). Cognitive interference put endoscopists ‘under pressure … to go the extra mile’ (P5) to reach targets, which could lead to gaming and harmful externalities. One participant described they were made aware of targets ‘but they’re not shoved down our throats’ (P8) which reduced the pressure to consider gaming (table 2: S6).
Harmful effects
When you are getting performance figures … at times you’ve got to think are you doing this [behaviour] for your figures or are you doing it for the patient … when you do it more for the patient, then you do notice your figures drop. So it is a hard one, to manage that. (P12)
Harmful effects are summarised in figure 2 in red and were mapped to a ‘gaming’ (P3) theme. Themes describing ‘harm’ were categorised as being indirect and direct. Indirect harms were generated from inaccurate documentation (‘fudging’ (P8)), and direct patient harms from removing polyps without clinical indication and persevering to complete procedures.
Inaccurate documentation: withdrawal time
A minimum withdrawal time of 6 min is set by the British Society of Gastroenterology.20 Withdrawal time was perceived not to be taken seriously by some endoscopists who would document 6 min without accurately noting the time. ‘“So, for the purposes of a quiet day I’m going to say this is six minutes and I really don’t care if anyone around me knows it isn’t”… I’m sure it happens in every department’ (P5).
In three centres, nursing assistants were trained to time withdrawal on behalf of endoscopists with the goal of improving withdrawal time as demonstrated in previous trials.34 This was initially perceived as intimidating external scrutiny by participants but they had come to consider that it reduced fudging withdrawal times. ‘I would like to think I wouldn’t [game withdrawal time] but it is hard for me to hide now because the nurses are documenting it … [Laughter]’ (P15). When assistants timed withdrawal, participants described other endoscopists engaging in time-wasting behaviours, such as starting the timer early and ‘hanging around’ in the rectum at the end of the test, these behaviours prolong the length of the test without improving colonic inspection or benefiting the patient (table 3: S1).
Harmful effects—documentation subthemes and illustrative quotations
Most participants expressed beliefs that polyp detection is important and longer withdrawal times improve detection. Participants assumed that endoscopists who undertake time-wasting behaviours did not appreciate the clinical importance of withdrawing slowly, ‘a lot of people just see it as getting the scope out. And maybe aren’t as aware that it’s a really key part of the examination, especially if they trained quite a long time ago’ (P12).
Inaccurate documentation: completion rates
The participants described examples where bowel preparation and procedure documentation could artificially inflate completion rates. Participants reported if endoscopists were unable to complete a colonoscopy that some converted the procedure documentation from a colonoscopy to a shorter flexible sigmoidoscopy (table 3: S2). One participant noted that if the insertion was difficult, endoscopists may inaccurately document inadequate bowel preparation, ‘oh poor bowel prep, let’s just come out’ (P12), to later justify a low completion rate (table 3: S3). Bowel preparation was not perceived to be under the endoscopist’s control, ‘you can’t change bowel prep’ (P17), and inadequate preparation limiting colonoscopy quality was not perceived as the endoscopist’s fault (table 3: S3).
Comfort score and patient experience inaccuracy
The participants perceived patient experience and comfort are important to colonoscopy quality and to patients. Comfort is a recognised colonoscopy quality KPI and used in colonoscopy A&F practice.20 Comfort scores are an assessment of the patient’s experience by the endoscopist or nursing assistant, these were perceived as inconsistent and of variable quality (table 3: S4).
One participant described their experience as a trainer, reviewing a trainee endoscopist’s portfolio and their patient comfort scores. The participant noted that in all 230 procedures, all patients were documented as being comfortable and noted that this would not be possible. This ‘horrified’ the participant, ‘[the trainee] said, “Well that’s what the consultant’s put on the thing.”… it just wasn’t important to [them]’ (P7).
Harmful effects: patient care
Perseverance despite patient discomfort
Participants perceived that colonoscopy can be painful, and that persistent patient discomfort should limit colonoscopy (table 4: S1). Participants described being ‘frightened’ (P7) by their completion rate performance figures causing them to ‘drive on and cause [patients] discomfort and pain’ (P14) to achieve a complete test. One participant perceived pressure to have a high completion rate to achieve BCSP accreditation and described completing procedures with poor bowel preparation despite being aware the behaviour ‘was unsafe, I’m going to miss loads of pathology here’ (P12, table 4: S2).
Harmful effects—patient care subthemes and illustrative quotations
Unnecessary polypectomy
Detection and removal of colonic polyps was described as important by all participants and the ‘main goal’ (P12) of colonoscopy. Participants described polyp detection and polypectomy KPIs as incentivising the removal of clinically insignificant lesions such as rectal diminutive hyperplastic polyps (table 4: S3). International guidance does not recommend the removal of such lesions.35 36 This behaviour to increase the recorded detection rate was recognised as having no clinical benefit to the patient, ‘snipping those off isn’t going to help a patient’ (P8), and potentially increasing the risks of complications particularly in the ‘elderly and frail’ causing ‘more harm’ (P7). Removing or leaving a polyp was not always a clear decision, and assessing risk and pragmatism were recognised as being important (table 4: S4).
Discussion
Statement of principal findings
Our study is the first to explore paradoxical effects and potential harms of current A&F interventions in colonoscopy using a dark logic model based on FIT. Paradoxical effects included social norms reassuring underperformance and performance anxiety causing cognitive interference which impacted the meta-task of confidence. Participants described inaccurately completed documentation so that completion rate and withdrawal time targets appeared to be achieved. Harmful behaviours included perseverance with the colonoscopy procedure despite patient discomfort and unnecessary polypectomy.
Strengths and weaknesses of the study
We are the first to present a dark logic model for A&F, such models focus on paradoxical effects and harms of BCIs, and to the reader and researcher can feel relentlessly negative. Dark logic models, like ours, should be situated in a wider model for behaviour change, incorporating intended effects and benefits, to inform A&F practice.
The study team perceived harms would be a difficult topic to discuss; however, participants frankly discussed gaming behaviours. On reflection, the interviewer was an endoscopist, who recognised participants’ experiences and used the same language and references. The interviewer was acquainted with four participants through academic or clinical work and had previously received training from five participants. The interviewer was junior to the participants in age, his position as a trainee and experience. It is possible that this shared clinical background and, in some instances, prior acquaintance helped establish rapport, encouraging an open dialogue, although occasionally communication had vestiges of the trainer–trainee relationship.
Our purposive sampling of participants with a range of professional backgrounds and clinical experiences adds to the transferability of these findings to wider clinical contexts.37 Fewer participants were selected in sites 4–6 to fulfil sampling criteria. Although participants’ experiences of A&F across sites were similar, sites had different organisational contexts, including safety management approaches and clinical leadership training,38 39 which may impact perception of performance management. Reassuringly, data saturation of themes was maintained across sites.
In responding to correspondence and agreeing to be interviewed about performance, we may have a self-selected group of those with a personal interest in colonoscopy quality. Although the prevalence of gaming behaviours in endoscopists is unknown, examples of gaming were described by endoscopists across professional backgrounds and lengths of experience. Participants rarely described their own negative behaviours, but those of unnamed others. These were disclosed in a conversational tone, with an implied intention to prevent them. The findings of this work were presented locally to endoscopy colleagues, who confirmed they recognised these behaviours in their own practice, and the pressures to undertake them.
Strengths and weaknesses in relation to other studies
The Cochrane review found A&F interventions were modestly effective, but demonstrated high variation in effectiveness.1 Ivers et al describe a lack of understanding regarding how A&F works; they recommend barriers to A&F effectiveness, including interpretation of interventions by clinicians, should be explored.40 One barrier of organisational targets causing individuals to undertake paradoxical behaviours has been described in the English public health sector as gaming or ‘reactive subversion’.41 Another hypothesised barrier is poorly validated healthcare outcome data,42 causing a balance of harms and benefits from false-negative and false-positive ‘diagnoses’ of quality care.
Past research evaluating A&F barriers has focused on organisational effects and not individual behaviours. A&F work in blood transfusion has used empirical qualitative study and theory to address organisational barriers to intervention efficacy.43 Paradoxical organisational findings have included variation in how hospitals received, shared and responded to feedback,44 and worsening variation in performance by applying action in an on-off manner.45
Our paper focuses on individual practitioner behaviours. Application of behavioural theories, such as clinical performance feedback intervention theory (CP-FIT), has been used to retrospectively explain why feedback may not have been effective at changing individuals’ behaviour, but without prospective theorisation of potential harms or paradoxical effects.46 Our paper demonstrates a theoretical model to prospectively hypothesise and explore A&F’s pathways to harmful effects from individual behaviours. This analytical process did not balance pathways of intended and unintended effects, but explored mechanisms of potential harms, prior to the implementation of an A&F intervention.
FIT and the TPB complimented each other as working theories in the analysis; TPB aided exploration of control and normative beliefs impacting intention, and FIT allowed mapping of their potential impact on behaviours. Exploration of paradoxical effects was enriched with normative beliefs. Social norm feedback is effective in changing healthcare behaviours when practitioners see themselves as an outlier, such as reducing antibiotic overprescribing.47 Psychology literature has described social norms having paradoxical ‘boomerang’ effects on high performers.48 Our participants did not describe boomerang effects from social comparisons, perhaps as there is little ambiguity that high detection is positive.49 However, low performers were reassured by low-performing peers. As suggested in social comparison theory, performance aligning with others reduces the motivation to change behaviour. This highlights the importance of using an aspirational social comparison of comparable peers (box 1).50 51
Implications for avoiding negative impacts of A&F.
To reduce negative impacts of A&F:
Use aspirational social feedback.
Measure what is important accurately, including patient-reported experience.
Identify and address educational needs around behaviours and documentation.
Avoid cognitive interference, anxiety and reducing confidence through:
Action plans targeting task motivation behaviours.
Providing personal support and buddying.
Participants reported inaccurate documentation caused by A&F pressures, where endoscopists identified targets they perceived as important and wished to appear to reach them. Changing documentation to game process outcomes is a recognised unintended consequence of A&F in endoscopy.52 53 Our participants described choosing inaccurate documentation over undertaking behaviours to improve quality. This may be related to low perceived control of behaviours from competing time pressures or workload.54 We identified potential educational needs around behaviours to improve performance and documentation; educational interventions addressing these and supplementing A&F may be effective (box 1).22
Mechanisms and implications for clinicians or policy makers
Cognitive interference
Colonoscopy performance has been described as a complex psychomotor skill requiring higher cognitive tasks,55 and our participants described confidence (a meta-task belief) as a requirement. KPIs in healthcare often have high levels of complexity in their underlying tasks.56 FIT suggests receiving negative feedback can confront perceptions of the self and cause anxiety. This draws attention away from undertaking tasks and increases pressure on performance, called cognitive interference.24 Our participants’ anxiety was increased by underperforming against national guidelines.20 Participants were aware that JAG recommends stopping endoscopists performing endoscopy if underperformance is assessed to be unsafe57; with perceived personal, psychological and financial consequences. Participants described cognitive interference may pressurise endoscopists to perform gaming behaviours. To avoid harmful behaviours, A&F interventions need to address the underlying cognitive interference which drives these behaviours (box 1).58
Measuring performance
Cognitive interference and gaming pressures are highest on behaviours with outputs perceived as being inaccurate or unmeasured, which may be sacrificed to achieve measured targets.41 A challenge for A&F is to identify what is important and measure it well.
Increasing the accuracy of targets to better measure important behaviours can prevent incentivising harmful behaviour. This study demonstrates that underperformance against detection targets incentivises removal of insignificant polyps, potentially risking patient safety. Improving targets to focus attention on clinically significant polyps and linking A&F systems to polyp histology data may reduce harmful behaviours.23 59 60
Patient experience is a key aspect of healthcare quality. Comfort scores that are endoscopist reported are criticised, as patients and clinicians have different priorities around the healthcare experience.58 61 Our study demonstrates perceptions that patient comfort documentation is variable and sometimes inaccurate. In the poor recording of the patient experience, A&F processes potentially expose patients to the risk of discomfort as practitioners may prioritise achieving better measured performance targets. Assessment and recording of the patient experience with validated patient-reported experience measures, such as the ‘Newcastle ENDOPROM’ in endoscopy, may reduce this risk (box 1).58
Unanswered questions and future research
Our dark logic model suggests addressing cognitive interference and anxiety of underperformance is critical for reducing potential A&F harms. Goal setting and action planning in A&F, focusing on task motivation behaviours which practitioners can implement to improve performance, may reduce cognitive interference (box 1). Such action planning is a clear tenet of FIT,24 CP-FIT,46 and is recommended for intervention design in the Cochrane review.27 For example, in colonoscopy, task motivation behaviours to improve detection include increasing withdrawal time and turning the patient’s position on withdrawal.62 63 A colonoscopy BCI with action plans targeting these behaviours and supplemental educational material is being tested in the NED-APRIQOT study.23
An endoscopy A&F intervention targeting leadership training demonstrated improved centre-wide colonoscopy performance.39 Our study demonstrated that where there is underperformance in meta-task behaviours, such as low confidence, then addressing this is a complex social task. This is the challenging work of local clinical leaders. Our study suggests clinical leads should clearly identify their motivation to provide support and alleviate anxiety. Opportunities to be observed performing complex skills, or buddying, for those persistently underperforming may be used to explore understanding of targets, develop behaviours poorly assessed by KPI and bolster confidence (box 1).33 Further study of leaders’ experiences implementing support for those persistently underperforming, and identification of leaders’ training needs should be explored in future research.
Conclusion
This example of using a dark logic model to map adverse effects has been insightful in our endoscopy setting and can be applied to different clinical settings where A&F is used to improve performance. Our dark logic model highlighted that A&F interventions, in accordance to FIT, may create a mix of desired and adverse effects. Without a priori theorisation, evaluations may disregard potential harms. In this setting, improved patient experience measures may reduce harm. To address cognitive interference the motivation of feedback to support improvement should always be clear, with plans targeting specific task motivation behaviours and offering face-to-face support for meta-task behaviours such as confidence.
Data availability statement
Data are available upon reasonable request. All participants provided written consent for non-identifiable publication of transcript extracts and direct quotations from data. Data were accessed in conjunction with Newcastle University data security policy. The data sets generated and/or analysed during the current study are not publicly available due to possible identification of participants through triangulation but are available from the corresponding author upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study involves human participants and ethics approval was granted as part of the National Endoscopy Database Automated Performance Reports to Improve Quality Outcomes Trial (NED-APRIQOT). Qualitative interview study ethics approval was granted by the Newcastle University Ethics Committee (Ref 9521/2018). Participants gave informed consent to participate in the study before taking part.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @DrJamieC, @RashmiBhardwaj0
Contributors JC acted as guarantor, developed the qualitative methodology, interviewed participants, analysed and interpreted data and wrote the manuscript. RB-G and LS were major contributors to the qualitative methodology, checked transcripts and codes and were major contributors in writing the manuscript. MDR developed the NED-APRIQOT protocol, identified eligible NHS endoscopy centres and was a contributor in writing the manuscript. FFS was a major contributor to both the qualitative methodology and in writing the manuscript.
Funding This study was funded by the Health Foundation (695428).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.