Teaching Evidence Based Medicine in Surgical Education; the Challenges and Techniques in Training
M Griffin1, *, DJ Jordan2, A El Gawad2
Identifiers and Pagination:Year: 2016
Issue: Suppl-3, M8
First Page: 337
Last Page: 345
Publisher Id: MEDJ-3-337
Article History:Received Date: 28/06/2015
Revision Received Date: 17/09/2016
Acceptance Date: 17/09/2016
Electronic publication date: 27/12/2016
Collection year: 2016
open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution-Non-Commercial 4.0 International Public License (CC BY-NC 4.0) (https://creativecommons.org/licenses/by-nc/4.0/legalcode), which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.
Evidence Based Medicine integrates clinical expertise, best available clinical evidence, as well as patient’s values and preferences to manage the care of patients. Surgeons have traditionally performed surgery according to their mentor teachings, these techniques being passed down through several generations. Current surgeon culture must evolve to integrate EBM into their clinical practice. The knowledge and skills required for searching and appraising critical literature needs to be taught to enable surgeons to implement it effectively. Evidence based surgery (EBS) will encourage surgeons to apply the best up-to-date knowledge to find the most effective surgical management plan for their patients. Several methods of teaching EBS to surgical trainees have shown to be effective including workshops, small group discussions, lecture style teaching and courses involving a combination of techniques. Journal clubs have gained in popularity and provided excellent teaching environments for surgeons to learn critical appraisal. Recently EBM has been introduced into the undergraduate programme to provide young medical practitioners with a strong foundation in EBM competency, and a positive attitude towards applying EBM to clinical practice. In this review, we aim to provide an overview of the principles of EBM and the success and challenges of teaching methods to deliver EBM for the surgical field.
INTRODUCTION TO EVIDENCE BASED MEDICINE
Surgeons have traditionally performed procedures by adopting techniques from their teachers, using techniques that they were told were effective . However, this practice of seeing one and learning from the teacher’s and one’s own mistake is no longer acceptable with the new concept of evidence-based medical practice . Evidence based medicine (EBM) is the management of patients using an optimal integration of best research evidence with clinical expertise whilst respecting patient values (Fig. 1) . The essential elements of applying EBM by a surgeon to a clinical problem includes the ability to perform a literature search to find the best available evidence, appraise whether the information is valid, and use a combination of clinical expertise, up to date knowledge and consideration for patients values to diagnose and manage their patient effectively . The evidence used during EBM practice is obtained from published scientific research. To allow for easy comparative of evidence, a hierarchy of evidence has been formed as shown in Fig. (2), with systematic reviews of randomised control trials being considered to have the highest level of validity and lowest level of bias . The interest in EBM has evolved over the last decade and begun to populate the literature with evidence illustrating; (1) the large daily need for valid and quantitative information on diagnosis, therapy and intervention; (2) the inadequacy of traditional resources such as books to provide this information as they are out of date; (3) the large gap between diagnostic skills and clinicians up-to-date knowledge and clinical performance, which declines with experience; and (4) the insufficient time with individual patients for finding and evaluating the evidence or time each week for reading and study . Therefore, EBM aims to bridge the gap between what we know and what we apply, providing an opportunity to improve surgical outcomes for the patients and improving surgeon experience and satisfaction (Fig. 3) .
|Fig. (1). Triad of evidence based medicine.|
|Fig. (2). The hierarchy of evidence.|
|Fig. (3). The positive outcomes of evidence based medicine in surgery taken from .|
PRINCIPLES OF EVIDENCE BASED MEDICINE
The most commonly used definition of evidence-based practice (EBP) was described by Sackett in 1996 as the “conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient. It means integrating individual clinical expertise with the best available external clinical evidence from systematic research” .
Firstly, the clinician needs to define a clinical question related to their clinical problem, for example ‘Do magnets help control pain in patients with osteoarthritis of the hip?’ To ensure, the question is focused and clear, clinicians can create questions to include the following components, patient or population, intervention, comparison of intervention and outcome, which can be remembered using the mnemonic PICO (Table 2). Secondly, the clinician will need to select the appropriate resources to conduct a search for this information. To obtain the primary literature clinicians will often use primary databases such as PubMed/MEDLINE. However, more recently clinicians have the option to use ‘pre-appraised resources’ or Evidence Based Practice (EBP) resources such as American College of Physicians (ACP) Journal Club, Clinical Evidence, Essential Evidence or UpToDate (Table 3). These resources review the original articles for expertise and applicability to clinical practice, providing the clinician with current and evidence based information to better inform their decisions. The Cochrane library is of particular use as it provides access to systematic reviews, which helps summarize the results from a number of studies. The third step is to appraise the information for validity and applicability and whether surgeons can apply the evidence to their individual patients. This is the step, which is often the most challenging to clinicians without prior any form of training. Though there are several tools and worklists to critically appraise the literature, there are several elements that are consistent when assessing the validity of studies. This can be illustrated using the PICO format described earlier. Firstly, the population requires consideration of how they were recruited and evaluation whether the population was an appropriate target population. Secondly, the intervention should be assessed for example; ‘how was the treatment allocated?’ Thirdly, the outcomes should be considered for their element of bias and measure of effectiveness. After effective appraisal the clinician will need to complete the fourth step, the integration of the data with their individual patient. The clinician must use their expertise to guide the integration of the data they have obtained. The final step is self-evaluation, evaluating their decision with their patient based on evidence-based practice.
|Ask||Convert the need for information into an answerable question|
|Acquire||Select appropriate resources to conduct a search of the literature|
|Appraisal||Critically appraise the evidence to assess for its validity|
|Integrate||Integrate the critical appraisal with our clinical expertise and with patients unique value and circumstances|
|Evaluate||Assess the effectiveness and efficiency of executing steps 1-4 for the next patient.|
|P||Patient, population, predicament or problem|
|I||Intervention, exposure, test or other agent|
|C||Comparison intervention, exposure or test|
|O||Outcome of clinical relevance, including time when it is relevant|
|ACP Journal Club||Online database since 2003, which contains quality-assessed, clinically rated original studies and reviews from over 130 clinical journals.|
|Clinical Evidence||Online international database that has developed rigorous systematic reviews to assess the quality and effectiveness of different treatments.|
|Dynamed||A unique online tool that aims to offer clinicians the ability to synthesize the evidence and provide objective analysis in an easily digestible format.|
|Essential Evidence||Evidence based tool that provides access to over 13000 topics, guidelines and summaries to provide up to date information on a variety of conditions.|
|FPIN Clinical Inquiries||Provides critical appraised and peer reviewed evidence to a number of clinical questions.|
|UpToDate||Online resource that provides clinicians with easy access to peer reviewed evidence to make correct decisions regarding patient care.|
TEACHING EVIDENCE BASED MEDICINE
Two approaches have been taken to improve the translation of evidence-based medicine into the clinical practice. Firstly it seems logical to provide the clinician with the tools to practice EBM during medical school, allowing newly qualified doctors to answers clinical problems in a systematic and effective manner, and to develop their EBM skills from the outset [6-11].
Various modules and courses have been implemented into the medical curriculum worldwide to teach EBM. For example a workshop delivered to third year medical students illustrated they were capable of using EBM resources and techniques following this intensive teaching experience . In addition, a randomized control trial illustrated an EBM course lasting one semester, in the 5th year of a public medical school in Mexico, and showed medical student training in EBM produced higher scores in attitudes, knowledge and self reported critical appraisal skills . Similarly, Hassanien et al illustrated that a student-selected module enabled fourth year medical students to gain a strong foundation in EBM skills for lifelong learning . Few studies have illustrated that integrating EBM into the medical curriculum is beneficial as opposed to teaching as it is a separate discipline . Tamin et al. illustrated that modifying the students preclinical years with EBM lectures and small groups sessions enabled students to learn early the importance of EBM in a physician’s career . Wanvarie et al. similarly integrating EBM longitudinally throughout the medical curriculum with students showing they were capable of completing the five steps by reporting high grades on EBM individual assignments and self evaluation . Liabsuetrakul et al. illustrated that the integration of EBM using small group discussion with case scenarios and problem-based learning enhanced their knowledge and skills of EBM in medical students .
Evidence suggests that an early, phased and integrated EBM is beneficial but the methods to teach EBM are still being developed. A recent randomized control trial has recently compared the efficacy of teaching EBM via didactic lectures and by structured case conferences to final year medical students . Weekly EBP-structured case conferences focusing on students' primary care patients significantly enhanced their knowledge and personal applications scores . Workshops have similarly shown to increase the skills to construct a clinical question compared to didactic teaching . Recently, a systematic review studied EBM teaching practices for undergraduate students and found the studies lacked robust outcome measures, urging more uniformity in assessment to ascertain the efficacy of interventions .
A second approach to enhancing EBM in surgery is teaching at the postgraduate level, particularly as EBM has only recently started to become introduced into the medical curriculum. Traditionally postgraduate teaching has been taught using courses away from the surgeon’s clinical practice. The use of seminars and tutorials has shown to improve the knowledge of postgraduate trainees . One-on-one teaching with tutorials has also shown to improve the knowledge in EBM and postgraduate reading quality . Half-day appraisal skills workshops orientated around problem based ideas in small groups has also shown to enhance EBM knowledge . Sprague et al. found that an intensive 2.5 day workshop with interactive lectures and small group sessions enabled significant improvement in the knowledge of surgical trainees . Several structured programs have been formulated to encourage EBM into surgery education, which may include some or all of the following formats; tutorials, one and one sessions, small group discussions, lectures and journal clubs [19, 20].
Journal clubs are events where clinicians discuss research articles and teach critical appraisal skills. and are becoming a well utilised and popular method of teaching EBM . Ahmad et al. evaluated an evidence-based review in surgery program to teach critical appraisal skills to general surgeon residents and surgeons across Canada . Seventy-five percent indicated they know about EBS due to teachings from the course with most respondents stating the journal clubs were valuable to their education . Similar findings were reported among general surgery and neurosurgery trainees. Temple et al. described a monthly journal club followed by self-directed assignments for one year and found it enhanced the critical appraisal skills and knowledge of epidemiology and biostatistics of plastic surgery residents .
A recent systematic review, in 2012 has tried to review the optimal method for teaching postgraduate surgical trainees. Only 7 articles were related to the teaching of EBM to surgical trainees, of which 4 reviewed the effectiveness of their curricula including 1 RCT and 3 before and after studies. Three reviewed participant satisfaction and one assessed changes in the participant’s attitudes to the effectiveness of EBM curriculum . The RCT performed by Kulier and colleagues illustrated that though a clinically integrated e-learning course with a lecture format increased the baseline knowledge, this was not significant and there were no changes in attitudes towards using EBM . However, Fritsche et al. demonstrated that a 3 day course to general surgery doctors enhanced their basic EBM knowledge and skills, from 6.3+/-2.9 at baseline to 9.9 +/-2.8 postcourse (p <0.001) . Khan et al. illustrated a journal club for obstetrics and gynecology surgeons enhanced their critical appraisal skills, increasing their reading time and number of articles read per week after the course . Similarly, the three studies which evaluated satisfaction used journal club teaching formats, which all illustrated participants finding they were able to develop their critical appraisal skills and statistical concepts. However, though journals clubs seem to be favoured by participants it still remains unclear which parts of journal club are the most important in learning . A randomized control trial involving 81 Canadian surgeons found that internet based journal club has also been shown to improve reading among general surgeons compared to self directed reading . Similarly to undergraduate study evaluation the authors concluded that with the small number of studies to evaluate the effectiveness of EBM teaching techniques to postgraduate surgeons, and further study is required .
A few studies have illustrated that effective EBM programs should be clinically relevant to ensure adequate uptake of skills. Toedter et al. designed an EBS teaching schedule that enabled the surgeons to develop EBS skills that they can use in their clinical practice . During the programme the surgeon is given a question they might be asked during a surgery round . Similarly Haines et al. described an EBM programme for neurosurgeons, which consisted of two hours set aside every other week to replace traditional professors' rounds with sessions led jointly by a neurosurgeon and an epidemiologist, but based on case presentations from patients currently being treated . In addition Grad et al used a programme, which consisted of eight one hourly weekly sessions with first two sessions involving lectures but the other sessions were based on problems encountered in clinical practice . Skills, attitudes and behaviour towards EBM was found to increase using self assessment .
Both approaches to teaching EBS are important to ensuring a positive impact of EBM on surgical patient care with recent systematic reviews proving that EMP competency increases regardless of whether EBM is delivered at an undergraduate or postgraduate level . However, the same review demonstrated that no study has directly compared teaching EBM to an undergraduate and postgraduate audience, demonstrating further evidence is required to determine if the timing is vital on clinicians competency and behaviour to adopting EBM tools to enhance patient care .
BARRIERS TO UPTAKE OF EBM BY SURGEONS
Improvements in evidence-based behaviour can only be increased if surgeons have a positive attitude toward, and have an increased awareness of, EBM. However, there are several other challenges that may act as potential barriers to ensuring the teaching of EBM to surgeons.
Firstly, the time at which EBM is delivered has been suggested to offer different challenges to the effective outcomes of EBM in surgery . At the postgraduate level health professionals time and awareness have shown to be the main barriers in gaining EBM competency . Whereas at an undergraduate level, a perceived lack of relevance to their learning has shown to be a barrier to uptake EBM knowledge and behaviour .
Few studies have tried to identify the potential challenges to implementing EBS. A qualitative study identified cultural factors the effected the implementation of evidence based medicine in surgical practice among Australian Surgeons. In depth interviews of 22 surgeons illustrated that although they had a good understanding of EBM they viewed it as not applicable to their actual clinical practice. The authors highlighted that EBM training should occur early in the surgeon’s career, be role modelled by the surgeons who exhibit a style of surgeon practice and should be workplace instructed to overcome these perceived barriers to ensure successful acceptance of EBM in surgery .
Similarly, surgical residents from McMaster University in Ontario illustrated that though most trainees wished to incorporate EBM into their clinical practice, time and inefficient processes to find appropriate information during patient time were barriers to achieving this . In addition, Mittal et al. found that though surgeons felt EBM improved patient care education in particular poor understanding of statistics and the surgery culture would prevent the uptake of EBM in their clinical practice .
A particular challenge for delivering EBM to surgeons is providing education to a practicing surgeon who would find participation in face-to-face conferences on a regular basis difficult. To overcome this it is important that the local workplace offers appropriate journals clubs, grand rounds, and courses on EBS, to ensure the uptake of EBM into clinical practice. Furthermore doctors should have access to medical library facilities and guidelines or protocols.
Some studies have highlighted that incorporating evidence based medicine into practice will be difficult due to the poor quality of studies in surgery, which are difficult to generalise to patient populations [37, 38]. Problems with surgical trials have been eluded due to several factors including difficulty in blinding, lack of randomized control trials due to ethical issue and learning curves illustrating that seniority usually results in better outcomes .
The availability of pre-appraised or aggregate evidence has shown to be a major factor to apply EBM. A questionnaire distributed to members of the Members of the Dutch Orthopaedic Association illustrated that EBM was welcomed by orthopaedic trainees. The development and use of evidence-based resources as well as preappraised summaries such as the journal's evidence-based medicine abstracts and Cochrane reviews were perceived as the best way to move from opinion-based to evidence-based orthopaedic practice .
A cross sectional study surveyed doctors and nurses in five major specialties to find interventions to improve evidence-based behaviour . Several potential solutions by doctors were found from the study including the enhancement of communications of evidence at meetings including ward rounds and handovers . The support from management was believed to be important in enhancing the uptake of EBM into clinical practice with suggestions of recruiting EBP minded leadership and having EBM role models for younger surgeons . Management could also support trainees by monitoring of EBM with yearly performances meetings for EBP activities .
THE FUTURE OF EBM IN SURGERY EDUCATION
It has now become accepted for surgeons to include evidence-based practice into their clinical-decision making. EBS is no longer just for senior academic surgeons but for all surgeons and patients. EBM is firmly part of clinical governance and the basis for the development of guidelines and frameworks suggested by the surgical colleges, National Institute for Clinical Excellence (NICE) and the World Health Organisation (WHO).
Teaching programmes needs to provide the foundation of EBS, which may be through case discussions, which are pertinent to the surgeon’s clinical situation. Integrating evidence based medicine into the ward rounds or clinic is another approach. To ensure the evolving field of surgical research is incorporated into practice the most effective techniques to teach EBM to postgraduate surgeons needs to be found. Surgeons must also take control of ensuring the good uptake of EBS into clinical care. Surgeons should support local hospital programmes, the support from library staff and become actively involved in collaborating management teams into enhancing EBS.
There are many platforms already present for the uptake of EBM. From the daily ward round, introduction of journal clubs and audit meetings, and attendance on workshop days, as well as national and international conferences. The latter provide a forum for discussion and an opportunity to utilise EBM in creating surgical guidelines.
With the emerging literature concerning EBM, the undergraduate curriculum providing the EMB tools to younger surgeons, the use of EBM will be more evident in the future. However, with better-designed studies evaluating the outcomes of different teaching methods and approaches, a better-integrated programme will be achieved.
Despite the challenges of introducing EBS at the undergraduate and postgraduate level, there is positive support from current surgeons in the utilisation of EBM tools to enhance patient care. The modern day surgeon does need to be aware of EBM. Implementation into their personal practice can only enhance their work and patient care. Involvement in journal clubs, subscription to journals and accessing surgical college forums and conferences are but a few ways to achieve these goals.
With future studies evaluating methods to tailor postgraduate teaching programs to ensure the utilisation of EBM in the clinical setting and formulation of integrated phased undergraduate EBM programs, patient outcomes will be improved.
CONFLICT OF INTEREST
The authors confirm that this article content has no conflict of interest.
|||Ashrafian H, Sevdalis S, Athanasiou T. Evidence Based Surgery. In: Key Topics in Key Topics in Surgical Research and Methodology. Springer Berlin Heidelberg 2010.
|||Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isnt. BMJ 1996; 312(7023): 71-2.
|||Bhandari M. Evidence-based medicine: why bother? Arthroscopy 2009; 25(3): 296-7.
|||Straus SE, Glasziou P, Richardson WS and Haynes RB Evidence based medicine How to practice and teach it. 4th ed. London: Churchill Livingstone 2011; pp. 1-7.|
|||Duke University Medical Center Library and the Health Sciences Library at the University of North Carolina at Chapel Hill. Introduction to evidence-based medicine. Available at: http://guides.mclibrary.duke.edu/ebmtutorial Accessed February 1, 2013.|
|||Sastre EA, Denny JC, McCoy JA, McCoy AB, Spickard A III. Teaching evidence-based medicine: Impact on students literature use and inpatient clinical documentation. Med Teach 2011; 33(6): e306-12.
|||Sánchez-Mendiola M, Kieffer-Escobar LF, Marín-Beltrán S, Downing SM, Schwartz A. Teaching of evidence-based medicine to medical students in Mexico: a randomized controlled trial. BMC Med Educ 2012; 12: 107.
|||Hassanien MA. Introduction to Evidence-Based Medicine: a student-selected component at the Faculty of Medicine, King Abdulaziz University. Adv Med Educ Pract 2011; 2: 215-9.
|||Tamim HM, Ferwana M, Al Banyan E, Al Alwan I, Hajeer AH. Integration of evidence based medicine into a medical curriculum. Med Educ Online 2009; 14: 15.
|||Wanvarie S, Sathapatayavongs B, Sirinavin S, Ingsathit A, Ungkanont A, Sirinan C. Evidence-based medicine in clinical curriculum. Ann Acad Med Singapore 2006; 35(9): 615-8.
|||Liabsuetrakul T, Sirirak T, Boonyapipat S, Pornsawat P. Effect of continuous education for evidence-based medicine practice on knowledge, attitudes and skills of medical students. J Eval Clin Pract 2013; 19(4): 607-11.
|||Cheng HM, Guo FR, Hsu TF, et al. Two strategies to intensify evidence-based medicine education of undergraduate students: a randomised controlled trial. Ann Acad Med Singapore 2012; 41(1): 4-11.
|||Ilic D. Teaching evidence-based practice: perspectives from the undergraduate and post-graduate viewpoint. Ann Acad Med Singapore 2009; 38(6): 559-.
|||Maggio LA, Tannery NH, Chen HC, ten Cate O, OBrien B. Evidence-based medicine training in undergraduate medical education: a review and critique of the literature published 20062011. Acad Med 2013; 88(7): 1022-8.
|||Gehlbach SH, Bobula JA, Dickinson JC. Teaching residents to read the medical literature. J Med Educ 1980; 55(4): 362-5.
|||Green ML, Ellis PJ. Impact of an evidence-based medicine curriculum based on adult learning theory. J Gen Intern Med 1997; 12(12): 742-50.
|||Ibbotson T, Grimshaw J, Grant A. Evaluation of a programme of workshops for promoting the teaching of critical appraisal skills. Med Educ 1998; 32(5): 486-91.
|||Sprague S, Pozdniakova P, Kaempffer E, Saccone M, Schemitsch EH, Bhandari M. Principles and practice of clinical research course for surgeons: an evaluation of knowledge transfer and perceptions. Can J Surg 2012; 55(1): 46-52.
|||Kwaan MR, Melton GB. Evidence-based medicine in surgical education. Clin Colon Rectal Surg 2012; 25(3): 151-5.
|||Honey CP, Baker JA. Exploring the impact of journal clubs: a systematic review. Nurse Educ Today 2011; 31(8): 825-31.
|||Ahmadi N, Dubois L, McKenzie M, Brown CJ, MacLean AR, McLeod RS. Role of Evidence-Based Reviews in Surgery in teaching critical appraisal skills and in journal clubs. Can J Surg 2013; 56(4): E98-E102.
|||Temple CL, Ross DC. Acquisition of evidence-based surgery skills in plastic surgery residency training. J Surg Educ 2011; 68(3): 167-71.
|||Ahmadi N, McKenzie ME, Maclean A, Brown CJ, Mastracci T, McLeod RS. Teaching evidence based medicine to surgery residents-is journal club the best format? A systematic review of the literature. J Surg Educ 2012; 69(1): 91-100.
|||Kulier R, Coppus SF, Zamora J, et al. The effectiveness of a clinically integrated e-learning course in evidence-based medicine: a cluster randomised controlled trial. BMC Med Educ 2009; 9: 21.
|||Fritsche L, Greenhalgh T, Falck-Ytter Y, Neumayer HH, Kunz R. Do short courses in evidence based medicine improve knowledge and skills? Validation of Berlin questionnaire and before and after study of courses in evidence based medicine. BMJ 2002; 325(7376): 1338-41.
|||Khan KS, Dwarakanath LS, Pakkal M, Brace V, Awonuga A. Postgraduate journal club as a means of promoting evidence-based obstetrics and gynaecology. J Obstet Gynaecol 1999; 19(3): 231-4.
|||Macrae HM, Regehr G, McKenzie M, et al. Teaching practicing surgeons critical appraisal skills with an Internet-based journal club: A randomized, controlled trial. Surgery 2004; 136(3): 641-6.
|||Toedter LJ, Thompson LL, Rohatgi C. Training surgeons to do evidence-based surgery: a collaborative approach. J Am Coll Surg 2004; 199(2): 293-9.
|||Haines SJ, Nicholas JS. Teaching evidence-based medicine to surgical subspecialty residents. J Am Coll Surg 2003; 197(2): 285-9.
|||Grad R, Macaulay AC, Warner M. Teaching evidence-based medical care: description and evaluation. Fam Med 2001; 33(8): 602-6.
|||Ilic D. Teaching evidence-based practice: perspectives from the undergraduate and post-graduate viewpoint. Ann Acad Med Singapore 2009; 38(6): 559-.
|||Coomarasamy A, Khan KS. What is the evidence that postgraduate teaching in evidence based medicine changes anything? A systematic review. BMJ 2004; 329(7473): 1017.
|||Jadad A. Randomised Controlled Trials. London: BMJ Publishing 1998.|
|||Kitto S, Petrovic A, Gruen RL, Smith JA. Evidence-based medicine training and implementation in surgery: the role of surgical cultures. J Eval Clin Pract 2011; 17(4): 819-26.
|||Bhandari M, Montori V, Devereaux PJ, Dosanjh S, Sprague S, Guyatt GH. Challenges to the practice of evidence-based medicine during residents surgical training: a qualitative study using grounded theory. Acad Med 2003; 78(11): 1183-90.
|||Mittal R, Perakath B. Evidence-based surgery: knowledge, attitudes, and perceived barriers among surgical trainees. J Surg Educ 2010; 67(5): 278-82.
|||McLeod RS. Instilling a culture of evidence-based surgery in Canada. World J Surg 2007; 31(8): 1551-5.
|||Jones RS, Richards K, Russell T. Relative contributions of surgeons and decision support systems. Surg Clin North Am 2006; 86(1): 169-179, xi. [xi].
|||Stirrat GM. Ethics and evidence based surgery. J Med Ethics 2004; 30(2): 160-5.
|||Poolman RW, Sierevelt IN, Farrokhyar F, Mazel JA, Blankevoort L, Bhandari M. Perceptions and competence in evidence-based medicine: Are surgeons getting better? A questionnaire survey of members of the Dutch Orthopaedic Association. J Bone Joint Surg Am 2007; 89(1): 206-15.
|||Ubbink DT, Vermeulen H, Knops AM, et al. Implementation of evidence-based practice: outside the box, throughout the hospital. Neth J Med 2011; 69(2): 87-94.