Abstract
Clinical audit is mandatory in the NHS
(National Health Service) in UK and mostly carried out by junior
medical staff as part of their training. However there has been
growing concern over whether this fulfils its intended purpose.
One of the important pitfalls in successful completion of an
audit is poor loop closure. This in turn fails to measure the
change in local practice and contributes to failure of the
system. We audited all audits carried out in our orthopaedic
department in the past 5 years to identify the flaws in the
system. We found poor loop closure (5%) and failure to change
practice based on audit (26%) as the major pitfalls in system.
We recommend that junior doctors should be encouraged to do a
re-audit and close an existing loop before starting a new topic
and that progress on all audits should be discussed in the
following meeting. We also suggest periodic audit of the audit
process by individual departments.
Keywords:
Clinical audit, training, orthopaedics.
J.Orthopaedics
2006;3(4)e5
Introduction:
The definition of clinical audit as
endorsed by NICE (National Institute of Clinical Excellence,
United Kingdom) is that ‘Clinical audit is a quality improvement
process that seeks to improve the patient care and outcomes
through systematic review of care against explicit criteria and
the implementation of change. Aspects of the structures,
processes and outcomes of care are selected and systematically
evaluated against explicit criteria. Where indicated, changes
are implemented at an individual team, or service level and
further monitoring is used to confirm improvement in healthcare
delivery'. It states that a clinical audit must include a review
date and identify the individual or individuals responsible for
their implementation. It also recommends that 90% of audits with
an action plan should be re-audited.
Participation in clinical audit is
mandatory for all clinical staff in the NHS in United Kingdom
(McCarthy, 1997). This is intended to improve the junior
doctor's concept about the process of audit and its relevance in
clinical practice. This is important part of their training
curriculum, so as to establish its importance in clinical
practice (Currie, 1998). Whether this serves its intended
purpose is debatable. Johnston et al (2000) in a study of 145
clinicians found that only 24% changed their practice based on
the results of an audit. Junior doctors understand the relevance
of clinical audit but are not allocated appropriate study
periods for carrying out these audits (McCarthy, 1997). Clinical
audit is a cycle of events leading to change in practice
depending on the findings and their relevance. Completion of
audit cycle is the final step in the process and this is usually
poor among participants (Lough, 1995).
Auditing the audit process itself improves
the process by identifying pitfalls in the cycle (Tabandeh,
1995). We audited the audit projects done in our present
department to get insight into how this system could be
improved.
Materials and methods
All orthopaedic audits
carried out in our hospital from 1999 to 2004 were reviewed
retrospectively. We looked at the audit topics, leads, whether
the audit was completed and presented, whether the audit loop
was closed and whether this has changed the local practice. The
database maintained by the clinical audit department was used to
extract the data into a structured data collection form. Where
data was found to be inadequate, attempts were made to contact
audit leads to get more details. The data was transferred to
Microsoft Excel for analysis.
Results
38 audits were done from
1999 to 2004 in our department. We found a recent increase in
the number of audits (10 in 2004 compared to 2 in 2001). 54 % of
audits were done by Senior House Officers and 32 % by registrars
(86 % of total number being done by junior doctors). Most (47 %)
audits looked at trauma related topics, investigations such as
radiography were reviewed in 16 % and 11 % looked at
administrative issues like case note completion. 11 % of audits
were not completed and in all of them the reason was that the
audit lead had left the hospital to join another one during the
project. In only 5% audit loop was closed by doing a re-audit.
We reviewed the current
local practice to see whether the suggestions from these audits
were being followed. The audits had successfully changed local
practice in 18 % and in another 16% the change was only partial;
only some of the recommendations were being followed. 26 % of
the audit projects had failed to change the practice. The rest
of the audits were either still active or it was too early to
assess outcome. The most popular topic was fractured neck of
femur (18%) followed by Fractures of distal radius (13 %) and
record keeping (10%). 3 of the audits were converted to a
regional or national presentation.
Discussion
Audit consumes a lot of
manpower in terms of working hours. Most audits are done by
junior doctors as part of their training. This consumes part of
their working hours or some make use of outside of work hours to
do audit depending on the pressure on them to complete the
audit. Most audits require review of case notes and audit
department usually pays for this. Some orthopaedic audits may
need collection of x-ray packets as well. Most hospitals have an
audit meeting once every 2, 3 or 6 months. The completed audit
projects are presented at this meeting and discussed. The need
for change in practice is usually considered here but may not
reach a consensus and some units do allocate the responsibility
to a consultant. The next audit meeting usually does not discuss
this topic again and unless someone does a re-audit at a later
date it is impossible to find out whether the practice has
changed at all. The whole process repeats at every meeting and
the attendance at these meetings is usually monitored as well.
Most hospitals cancel all elective work for audit meetings and
this improves attendance at these meetings (McCarthy, 1997). In
one study, 41 % clinicians felt that audit consumes resources
that could be better used on patient care (Smith, 1992). We had
a similar feeling when we found that 26% of the audits failed to
change the local practice. The authors have worked in different
NHS hospitals and have been part of many audits during our
career. With our experience from different hospitals, we knew
that this was endemic in NHS; but this should be considered as
anecdotal evidence only.
It is important to
complete audit cycle by doing re-audit. We found that only 5% of
orthopaedic audits were re-audited in our department. This
compromises the fundamental principle of audit and thereby
clinical effectiveness. Most audits are undertaken by junior
doctors who are keen to start an audit as soon as they start a
substantive six month training post. The topics are either
chosen by them or allocated to by their consultant. In an
attempt to project themselves forward, they rarely take up a
re-audit nor are they advised to do one. Most junior doctors in
Orthopaedics do one or two audit projects during a 6 month
appointment. We feel that the process may be strengthened by
allocating re-audits to junior doctors along with new topics.
Audit is a form of
retrospective research for most units. This allows them to do a
retrospective study without the hassle of going through the
lengthy process of ethics committee approval. This seems to be
the norm and usually reflects in the conversion rate of these
audits to regional or national presentations. This conversion
rate was low in our department (8%).This possibly can be
improved by doing a re-audit and proving that evidence from
audit had improved local practice. This should make the findings
relevant to a broader group than the respective department. This
generally varies between departments and is generally higher in
university hospitals.
Conclusions
We make the following
recommendations based on our findings:
-
Junior doctors should
be encouraged to do re-audits along with new projects.
-
Audits should identify
the person responsible for the planned change and the progress
made should be discussed in the following meeting.
-
Every department
should audit its own audits periodically to review changing
practice and identify defects in the system.
References
-
Currie IS,
Paterson-Brown S. (1998)
Clinical audit; who is auditing who? Scott Med J. 43(6):185-8.
-
Johnston G,
Davies HT,
Crombie IK. (2000) Improving
care or professional advantage? What makes clinicians do audit
and how well do they fare? Health Bull (Edinb). 58(4):276-85.
-
Lough JR,
Mckay J,
Murray TS. (1995) Audit:
trainers' and trainees' attitudes and experiences. Med Educ.
29(1):85-90.
-
McCarthy MJ,
Byrne GJ. (1997) Surgical
audit: the junior doctors' viewpoint.J R Coll Surg Edinb. 42(5):317-8.Principles for Best Practice in Clinical Audit. NICE. (2002)
Radcliffe Medical Press.
-
Smith HE,
Russell GI,
Frew AJ,
et al. (1992) Medical audit:
the differing perspectives of managers and clinicians. J R
Coll Physicians Lond. 26(2):177-80.
-
Tabandeh H,
Thompson GM. (1995) Auditing
ophthalmology audits. Eye. 9:1-5.
|