Basic anesthesia skills simulation curriculum for medical students

358 Pages·2006·1.84 MB·English

by  Anja Schmitz

RUPRECHT-KARLS-UNIVERSITÄT-HEIDELBERG
PSYCHOLOGISCHES INSTITUT

Basic anesthesia skills simulation curriculum for

medical students:

Development and empirical evaluation based on

an instructional design model

Anja Schmitz

Dissertation

zur Erlangung des akademischen Grades eines

Dr. phil.
der Fakultät für Verhaltens- und Empirische
Kulturwissenschaften

der Ruprecht-Karls-Universität Heidelberg

August 2006

Erstgutachter: Prof. Dr. Niclas Schaper

Zweitgutachter: Prof. Dr. Karlheinz Sonntag
ii

For my grandparents
iii

“No matter how a training
device is designed, no
matter what its level of
fidelity, it will not be an
effective trainer if it is not
used properly.”

Hays & Singer, 1989, p. 42
Acknowledgements iv
Acknowledgements

I want to take this opportunity to thank everybody who was to some extent involved in the
process of this dissertation.

First of all, I thank my advisor Prof. Dr. Niclas Schaper for his constructive feedback and
guidance throughout all stages of this project. I am especially indebted to him for assuring
that the project could be completed even after he took a position at the University of
Paderborn.

I also greatly appreciate the help of Prof. Dr. Karlheinz Sonntag, who served as the second
advisor. He provided me at the Psychology Institute with the necessary resources to complete
this project.

This dissertation would furthermore not have been possible without Prof. Dr. Bernhard Graf,
the HANS team and their support during the data collection process. Special debts are owed
to Dr. Christoph Grube and Dr. Thomas Boeker, who contributed their time and expertise.

The study would have been even less possible without the medical school students, whom I
want to thank for taking the time to fill out yet another set of questionnaires.

Thanks go out to my friends and colleagues, especially Christian Unkelbach and Christina
Röcke who provided helpful hints and feedback on drafts of this paper and who kept me
motivated throughout the process. I also appreciate Tetiana Stadnytska and Petra Hasselbach
taking the time to discuss some methodological issues with me.

I particularly want to thank Oliver Emmler who kept me focused through the final stages of
the project. He also provided very useful medical and IT helpdesk services to me.

Special thanks go to Dietmar Chur, Ute Fehr, and Petra Eggensperger (University of
Heidelberg Competence Center), who challenged and supported me in my professional
development and provided a lot of support on a personal level. Without the opportunity to
work at the Competence Center I could not have financed this project.

Most of all, I want to thank my family for their enduring love and support.
Abstract v
Abstract
During the last decades medical school curricula have been reformed to put more focus on
building students’ practical skills. In the course of these reforms new teaching methods were
employed, among them simulation. By now, simulation is a widely used training method,
especially in the area of anesthesia. It currently still suffers from two sets of problems, namely
the lack of a learning theoretical foundation and thorough evaluation studies. The current
study revised an existing basic anesthesia skills simulation curriculum based on an
instructional model for the development, delivery, and evaluation of medical simulation
training (Schaper, Schmitz, Graf & Grube, 2003). The original version of the training (TG 0)
was compared with two instructionally revised versions (TG 1 and 2). The revised curriculum
was designed following an integrated approach to learning (Reinmann-Rothmeier & Mandl,
2001). Accordingly, the didactic strategy of the simulation training’s preparatory seminar for
TG 1 and 2 was revised to combine constructivist (e.g., problem-based), and traditional (e.g.,
lecture-based) elements. The full-scale simulation sessions for TG 1 and 2 were revised based
on the cognitive apprentice-ship approach (Collins, Brown & Newman, 1989) to provide a
constructivist learning environment with instructional support. In addition to these changes,
the sequence of the curriculum’s elements was varied for TG 2. Before participating in the
full-scale complete-task simulation sessions, TG 2 participated in part-scale part-task
emergency simulation training.
The evaluation of the revised training programs yielded the following main results: The three
groups reacted positively to the training but did overall not differ in their reactions. The
revisions of the preparatory seminar did not lead to the expected increase in self-reported
learning. The revisions of the full-scale simulation sessions and the change in the
curriculum’s elements’ sequence led to more self-reported learning for TG 2. Ratings by the
instructors supported this result by showing that TG 2 was better prepared for the simulation
sessions. On the transfer level, assessed by the students’ OSCE results, TG 2 also received the
best ratings. A set of students’ individual characteristics and key relations between the
model’s elements were also investigated: Students’ readiness and expectation fulfillment were
positively influenced by respective interventions. Students’ readiness, expectation fulfillment,
instrumentality expectancy, and self-efficacy predicted their training reactions. The study thus
corroborated that simulation training is a viable method to build students’ basic anesthesia
skills. It provided first indications that a simulation curriculum based on a theoretically
founded model improves training effectiveness and that individual characteristics influence
simulation outcomes. Stricter designs are needed to extend the study’s findings.
Contents vi
Contents
ACKNOWLEDGEMENTS.............................................................................................................IV
ABSTRACT..................................................................................................................................V
CONTENTS.................................................................................................................................VI
LIST OF TABLES......................................................................................................................XIV
LIST OF FIGURES...................................................................................................................XVII
1 INTRODUCTION..................................................................................................................1
2 THEORETICAL AND EMPIRICAL BACKGROUND.................................................................6
2.1 THEORETICAL APPROACHES TO LEARNING AND INSTRUCTION.....................................6
2.1.1 Cognitivist learning theories..............................................................................6
2.1.1.1 General concepts............................................................................................6
2.1.1.2 Instructional Design as a specific cognitivist approach.................................7
2.1.1.3 Critique of the cognitivist approach...............................................................8
2.1.2 Constructivist learning theory............................................................................8
2.1.2.1 General concepts............................................................................................9
2.1.2.2 Anchored instruction as a specific constructivist approach.........................11
2.1.2.3 Critique of the constructivist approach........................................................12
2.1.3 Integration of cognitivist and constructivist approaches to learning and
instruction.........................................................................................................14
2.1.3.1 Knowledge-based constructivism – an integrated approach to learning......14
2.1.3.2 Cognitive apprenticeship as a specific integrated approach.........................15
2.1.4 Conclusion........................................................................................................17
2.2 THEORETICAL APPROACH TO THE INSTRUCTIONALLY-BASED DEVELOPMENT OF
SIMULATOR TRAINING: A MODEL OF SIMULATOR-TRAINING EFFECTIVENESS
(SCHAPER, SCHMITZ, GRUBE, & GRAF, 2003)...........................................................18
2.2.1 Description of the model..................................................................................19
2.2.1.1 Individual characteristics..............................................................................21
2.2.1.1.1 Training expectations and expectation fulfillment.................................21
2.2.1.1.2 Goal orientation......................................................................................22
2.2.1.1.3 Trainee readiness....................................................................................24
2.2.1.1.4 Self-efficacy...........................................................................................25
2.2.1.2 Training and transfer motivation..................................................................26
2.2.1.2.1 Valence-Instrumentality-Expectancy.....................................................28
2.2.1.3 Organizational characteristics......................................................................30
2.2.1.4 Instructional Systems Design (ISD).............................................................31
2.2.1.4.1 Training needs assessment.....................................................................31
2.2.1.4.2 Training objectives and content.............................................................33
2.2.1.4.3 Evaluation criteria..................................................................................34
2.2.1.5 Training Design............................................................................................34
2.2.1.5.1 Training design: Learning design...........................................................35
2.2.1.5.1.1 Instructor role..................................................................................35
2.2.1.5.1.2 Instructional strategy and methods..................................................38
2.2.1.5.1.3 Feedback..........................................................................................40
2.2.1.5.1.4 Training motivation.........................................................................43
2.2.1.5.1.5 Transfer support..............................................................................45
2.2.1.5.2 Training design: Simulator and simulation design.................................49
2.2.1.5.2.1 Simulator design..............................................................................49
2.2.1.5.2.1.1 Types of simulators..................................................................50
2.2.1.5.2.1.2 Fidelity of simulators...............................................................53
2.2.1.5.2.2 Simulation design............................................................................55
Contents vii
2.2.1.5.2.2.1 Scenario content.......................................................................55
2.2.1.5.2.2.2 Scenario sequence....................................................................56
2.2.1.6 Training outcomes........................................................................................57
2.2.1.6.1 Reaction/Training appraisal...................................................................57
2.2.1.6.2 Learning.................................................................................................58
2.2.1.6.3 Behavior/Transfer...................................................................................59
2.2.1.6.4 Organizational effects/Results................................................................61
2.2.1.6.5 Conclusion..............................................................................................61
2.2.2 Conclusion........................................................................................................63
2.3 EMPIRICAL FINDINGS: THE USE OF SIMULATORS IN ANESTHESIA TRAINING...............64
2.3.1 Simulators as training tools.............................................................................64
2.3.1.1 Simulators as training tools for physicians..................................................64
2.3.1.2 Simulators as training tools for residents.....................................................65
2.3.1.3 Simulators as training tools for students......................................................66
2.3.2 Simulators as assessment tools........................................................................69
2.3.3 Conclusion........................................................................................................70
2.4 CONCLUSION AND RESEARCH QUESTION OF THE PRESENT STUDY..............................72
3 THE PRESENT STUDY: INSTRUCTIONALLY BASED DESIGN AND EVALUATION OF A ...........
BASIC ANESTHESIA SKILLS SIMULATION CURRICULUM..................................................73
3.1 THE STUDY SETTING..................................................................................................73
3.1.1 The reformed medical school curriculum at the University of Heidelberg......73
3.1.2 The HANS Simulation Center...........................................................................74
3.1.3 Conclusion........................................................................................................76
3.2 DESCRIPTION OF THE ORIGINAL ANESTHESIOLOGY CURRICULUM..............................76
3.2.1 Preparatory introductory anesthesiology seminar..........................................76
3.2.2 Simulation sessions..........................................................................................77
3.2.2.1 Anesthesia full-scale simulation sessions....................................................77
3.2.2.2 Emergency medicine part-scale simulation sessions...................................77
3.2.3 Conclusion........................................................................................................78
3.3 DEVELOPMENT OF THE REVISED ANESTHESIOLOGY CURRICULUM.............................78
3.3.1 Instructional systems design tasks....................................................................78
3.3.1.1 Needs assessment.........................................................................................79
3.3.1.2 Learning objectives......................................................................................80
3.3.1.3 Evaluation criteria........................................................................................82
3.3.1.4 Training evaluation considerations..............................................................82
3.3.1.4.1 General evaluation design considerations..............................................83
3.3.1.4.2 Response-shift bias.................................................................................83
3.3.1.4.3 Conclusion..............................................................................................85
3.3.2 Training design tasks: Development of the interventions................................85
3.3.2.1 Design of the revised preparatory seminar...................................................86
3.3.2.1.1 An integrated approach to learning as the didactic basis for the
seminar’s instructional strategy..............................................................86
3.3.2.1.1.1 Fostering student activity................................................................89
3.3.2.1.1.2 Fostering student motivation...........................................................90
3.3.2.1.1.3 Fostering learning goal orientation.................................................91
3.3.2.1.1.4 Fostering readiness..........................................................................92
3.3.2.1.1.5 Fostering training expectations.......................................................93
3.3.2.1.1.6 Fostering transfer.............................................................................94
3.3.2.1.2 Conclusion..............................................................................................95
3.3.2.2 Design of the revised full-scale simulation sessions....................................95
3.3.2.2.1 Scenario development and scenario content..........................................96
Contents viii
3.3.2.2.2 An integrated approach to learning as the didactic basis of the
full-scale simulation sessions’ instructional strategy.............................97
3.3.2.2.2.1 Full-scale simulation sessions’ structure and content.....................97
3.3.2.2.2.2 Realization of constructivist design principles in full-scale
simulations....................................................................................100
3.3.2.2.2.3 Cognitive apprenticeship elements providing instructional support
for the learners...............................................................................101
3.3.2.2.3 Conclusion............................................................................................103
3.3.2.3 Design of the revised curricular sequence..................................................103
3.3.2.3.1 Redesign of the curricular sequence: Phase 1: Spacing changes.........103
3.3.2.3.2 Redesign of the curricular sequence: Phase 2: Order and spacing
changes.................................................................................................104
3.3.2.3.3 Conclusion............................................................................................106
3.3.3 Instructor training..........................................................................................106
3.3.4 Conclusion......................................................................................................107
3.4 METHOD OF THE EVALUATION STUDY.....................................................................108
3.4.1 Hypotheses.....................................................................................................108
3.4.1.1 Reaction level hypotheses..........................................................................108
3.4.1.2 Learning level hypotheses..........................................................................110
3.4.1.3 Transfer level hypothesis...........................................................................112
3.4.1.4 Individual characteristics hypotheses.........................................................112
3.4.1.5 Hypotheses about relations within the model............................................114
3.4.2 Study design....................................................................................................115
3.4.3 Evaluation instruments...................................................................................119
3.4.3.1 Evaluation questionnaires..........................................................................119
3.4.3.1.1 Student questionnaires..........................................................................120
3.4.3.1.1.1 Self-perceived anesthesia competencies.......................................120
3.4.3.1.1.2 Students’ training expectations, perceptions, and expectation
fulfillment......................................................................................121
3.4.3.1.1.3 Instrumentality expectancy...........................................................123
3.4.3.1.1.4 Situational goal orientation...........................................................124
3.4.3.1.1.5 Students’ readiness........................................................................125
3.4.3.1.1.6 Self-efficacy..................................................................................125
3.4.3.1.1.7 Reaction items...............................................................................126
3.4.3.1.1.7.1 General student reactions.......................................................127
3.4.3.1.1.7.2 Student reactions to curricular sequence................................127
3.4.3.1.1.7.3 Student reactions to instructor’s teaching skills.....................128
3.4.3.1.1.8 Demographics................................................................................129
3.4.3.1.2 Instructor questionnaires......................................................................130
3.4.3.2 Objective standardized clinical examinations (OSCE)..............................131
3.4.3.3 Resulting pre-/post- and transfer-test instruments.....................................132
3.4.4 Participants....................................................................................................133
3.4.4.1 Student participants....................................................................................134
3.4.4.2 Instructor participants.................................................................................134
3.4.5 Procedure.......................................................................................................134
3.4.5.1 Pretest.........................................................................................................134
3.4.5.2 Posttest.......................................................................................................135
3.4.5.3 Transfer-test...............................................................................................135
3.4.6 Analytic strategy.............................................................................................136
3.4.6.1 Data quality management...........................................................................136
3.4.6.2 Scale analysis.............................................................................................136
Contents ix
3.4.6.3 Hypothesis tests..........................................................................................138
3.4.6.3.1 Analyses of the response-shift..............................................................138
3.4.6.3.2 Analyses of the training effects............................................................139
3.4.6.3.3 Analyses of the relations within the model..........................................139
4 RESULTS OF THE PRESENT STUDY.................................................................................142
4.1 SAMPLE CHARACTERISTICS.....................................................................................142
4.1.1 Demographic information..............................................................................142
4.1.1.1 Number of participants, age, and gender....................................................142
4.1.1.2 Prior clinical experience.............................................................................143
4.1.2 Conclusion......................................................................................................145
4.2 TRAINING OUTCOMES..............................................................................................146
4.2.1 Level 1: Training reactions............................................................................146
4.2.1.1 Reactions to the full-scale simulation and the simulator............................146
4.2.1.1.1 Full-scale simulation training’s value for the curriculum....................146
4.2.1.1.2 Effects of the simulator’s realism.........................................................147
4.2.1.1.3 Simulator’s suitability as a training tool..............................................148
4.2.1.1.4 Adjustment to the simulation...............................................................149
4.2.1.1.5 Conclusion............................................................................................150
4.2.1.2 Reactions to the scenarios..........................................................................151
4.2.1.2.1 Scenarios’ suitability............................................................................151
4.2.1.2.2 Simulation relevancy............................................................................152
4.2.1.2.3 Conclusion............................................................................................153
4.2.1.3 Reactions to the revised curricular sequence.............................................154
4.2.1.3.1 Curricular sequence and coherence......................................................154
4.2.1.3.2 Conclusion............................................................................................154
4.2.1.4 Overall satisfaction with the full-scale simulation training.......................155
4.2.1.5 Reactions to the instructors........................................................................155
4.2.1.5.1 Instructors’ teaching skills...................................................................156
4.2.1.5.2 Conclusion............................................................................................160
4.2.2 Level 2: Learning...........................................................................................161
4.2.2.1 Preliminary analyses of the anesthesia competency scale’s psychometric
qualities......................................................................................................161
4.2.2.1.1 Examination of the anesthesia competency scale’s structure...............161
4.2.2.1.1.1 Investigation of the anesthesia competency scale’s factor
extraction.......................................................................................162
4.2.2.1.1.2 Investigation of the anesthesia competency scale’s item
adequacy: communalities..............................................................163
4.2.2.1.1.3 Investigation of the anesthesia competency scale’s item
adequacy: factor loadings..............................................................164
4.2.2.1.1.4 Investigation of the extracted anesthesia competency factors’
interpretability...............................................................................165
4.2.2.1.1.5 Investigation of the anesthesia competency factors’ internal
consistency....................................................................................167
4.2.2.1.1.6 Conclusion: Final factor solution of the anesthesia competency
scale...............................................................................................167
4.2.2.1.1.7 Intercorrelations among the anesthesia competency factors.........170
4.2.2.1.2 Changes regarding the students’ internal frame of reference...............170
4.2.2.1.2.1 Investigation of the occurrence of a response-shift.......................171
4.2.2.1.2.2 Discussion of changes regarding the students’ internal frame of
reference: Response-shift..............................................................174
Contents x
4.2.2.2 Self-reported learning: Training variation effect on students’
self-perceived anesthesia competency.......................................................175
4.2.2.2.1 Investigation of the overall training variation effect............................176
4.2.2.2.1.1 Descriptive and inferential analyses..............................................176
4.2.2.2.1.2 Conclusion.....................................................................................179
4.2.2.2.2 Investigation of the training groups’ seminar and simulation learning
gains.....................................................................................................180
4.2.2.2.2.1 Descriptive and inferential analyses..............................................180
4.2.2.2.2.2 Conclusion.....................................................................................182
4.2.2.2.3 Investigation of the seminar’s variation effect.....................................183
4.2.2.2.3.1 Descriptive and inferential analyses..............................................183
4.2.2.2.3.2 Conclusion.....................................................................................184
4.2.2.2.4 Investigation of the simulations’ variation effect.................................184
4.2.2.2.4.1 Descriptive and inferential analyses..............................................184
4.2.2.2.4.2 Conclusion.....................................................................................185
4.2.2.2.5 Exploratory investigation of the seminars’ and simulation sessions’
learning effects.....................................................................................186
4.2.2.2.5.1 Descriptive and inferential analyses..............................................186
4.2.2.2.5.2 Conclusion.....................................................................................187
4.2.2.3 Investigation of other-reported learning: Training variation effect in
instructor ratings.........................................................................................188
4.2.2.3.1 Descriptive and inferential analyses.....................................................188
4.2.2.3.2 Conclusion............................................................................................189
4.2.3 Level 3: Behavior...........................................................................................190
4.2.3.1 Investigation of students’ OSCE scores: Training variation effect............190
4.2.3.1.1 Descriptive and inferential analyses.....................................................190
4.2.3.1.2 Conclusion............................................................................................191
4.3 INDIVIDUAL CHARACTERISTICS...............................................................................192
4.3.1 Training expectations, perceptions, and expectation fulfillment...................192
4.3.1.1 Training expectations regarding active participation and knowledge
application..................................................................................................192
4.3.1.1.1 Examination of the training expectation scale’s structure...................193
4.3.1.1.1.1 Investigation of the training expectation scale’s factor extraction193
4.3.1.1.1.2 Investigation of the training expectation scale’s item adequacy:
communalities...............................................................................194
4.3.1.1.1.3 Investigation of the training expectation scale’s item adequacy:
factor loadings...............................................................................195
4.3.1.1.1.4 Investigation of the extracted training expectation factor’s
interpretability...............................................................................195
4.3.1.1.1.5 Investigation of the training expectations factor’s internal
consistency....................................................................................196
4.3.1.1.1.6 Conclusion: Final factor solution of the training expectations
scale...............................................................................................197
4.3.1.1.2 Investigation of the intervention effect on students’ training
expectations..........................................................................................197
4.3.1.1.2.1 Descriptive statistics of the students’ training expectations..........197
4.3.1.1.2.2 Inferential statistics of the students’ training expectations...........198
4.3.1.1.2.3 Conclusion.....................................................................................199
4.3.1.2 Training perceptions regarding active participation and knowledge
application..................................................................................................199
4.3.1.2.1 Examination of the training perceptions scale structure......................199