evaluating the appropriateness of electronic information resources for learning in pre-clerkship ...

181 Pages·2016·4.56 MB·English

by  Dinara Saparova

EVALUATING THE APPROPRIATENESS OF ELECTRONIC INFORMATION
RESOURCES FOR LEARNING IN PRE-CLERKSHIP MEDICAL EDUCATION:
AN EYE TRACKING STUDY
________________________________________________________

A Dissertation
presented to
the Faculty of the Graduate School
University of Missouri-Columbia

______________________________________________

In Partial Fulfillment
of the Requirements for the Degree
Doctor of Philosophy

________________________________________________________

by
DINARA SAPAROVA
Dissertation Supervisor – Dr. Joi L. Moore

DECEMBER 2015
The undersigned, appointed by the dean of the Graduate School, have examined the
dissertation entitled
EVALUATING THE APPROPRIATENESS OF ELECTRONIC INFORMATION
RESOURCES FOR LEARNING IN PRE-CLERKSHIP MEDICAL EDUCATION:
AN EYE TRACKING STUDY
presented by Dinara Saparova,
a candidate for the degree of Doctor of Philosophy,
and hereby certify that, in their opinion, it is worthy of acceptance.

________________________________________________________
Dr. Joi L. Moore (Chair)

________________________________________________________
Dr. Sanda Erdelez

________________________________________________________
Dr. Gail Fitzgerald

________________________________________________________
Dr. Suzanne Boren
To those
who have supported me
during this journey….
ACKNOWLEDGEMENTS

I am very lucky to have some of the brightest people on my dissertation
committee who have been providing valuable guidance and advice during the years of the
program and especially during the writing of this dissertation. I also could not have made
it without the tremendous support of my family, colleagues, and friends. My deepest
gratitude goes to my husband, my parents, and my University of Missouri colleagues.
Special thanks to Nathanial Nolan, Neeley Current, Lada Micheas, Dr. Kimberly
Hoffman, Dr. Michael Hosokawa, and Dr. Jeffrey Belden; all of my University of
Missouri friends; and finally to little Mathew who has shared with me nine months of the
dissertation writing journey and daily office time.

  ii
TABLE OF CONTENTS
ACKNOWLEDGEMENTS................................................................................................ii
LIST OF ILLUSTRATIONS..............................................................................................v
LIST OF TABLES............................................................................................................vii
ABSTRACT.....................................................................................................................viii
CHAPTER 1 – INTRODUCTION..................................................................................1
Problem Statement..........................................................................................................1
Purpose of the Study.....................................................................................................10
Research Questions.......................................................................................................11
Significance of the Study..............................................................................................14
Outline of Research Methods........................................................................................15
Assumptions, Delimitations, and Limitations of the Study..........................................16
CHAPTER 2 – LITERATURE REVIEW....................................................................18
Theoretical Perspectives on Learning in Hypermedia..................................................18
Principles of Cognitive Architecture and Cognitive Load Theory...............................21
Cognitive Load in Hypermedia Learning.....................................................................22
The Cognitive Theory of Multimedia Learning in Hypermedia Learning...................32
Measurements of Cognitive Load.................................................................................35
CHAPTER 3 – METHODS...........................................................................................45
Research Objective and Questions................................................................................45
Study Setting and Participants......................................................................................45
Materials and Instruments.............................................................................................46
Apparatus......................................................................................................................52
Procedure......................................................................................................................55
Data Analysis................................................................................................................62
CHAPTER 4 – RESULTS..............................................................................................74
Participants’ Demographics..........................................................................................74
Survey Results..............................................................................................................74
Research Question 1.....................................................................................................78

  iii
Research Question 2.....................................................................................................84
Research Question 3.....................................................................................................89
CHAPTER 5 – DISCUSSION.....................................................................................114
Research Question 1...................................................................................................114
Research Question 2...................................................................................................119
Research Question 3...................................................................................................121
Limitations, Implications, and Future Research.........................................................131
Conclusions.................................................................................................................135
APPENDICES................................................................................................................137
Appendix A: Demographic Information Survey.........................................................137
Appendix B: Preferences for Electronic Information Resources Survey...................139
Appendix C: Screening Test.......................................................................................140
Appendix D: Knowledge Tests...................................................................................141
Appendix E: Interview Protocol.................................................................................154
REFERENCES...............................................................................................................156
VITA...............................................................................................................................170

  iv
LIST OF ILLUSTRATIONS
FIGURE 1. NESTING OF NEW MEDIA....................................................................................19
FIGURE 2. TWO-STEP QUESTION ORGANIZATION................................................................48
FIGURE 3. TOBII X2-60 EYE TRACKER................................................................................53
FIGURE 4. CORNEAL REFLECTION AND BRIGHT PUPIL.........................................................54
FIGURE 5. TWELVE POSSIBLE REPLICATIONS OF 3 X 3 LATIN SQUARES..............................58
FIGURE 6. PERMUTATIONS OF THE THREE DISEASES...........................................................58
FIGURE 7. OVERVIEW OF THE EXPERIMENTAL PROCEDURE................................................61
FIGURE 8. AOIS IN UPTODATE..........................................................................................64
FIGURE 9. AOIS IN ACCESS MEDICINE................................................................................65
FIGURE 10. AOIS IN WIKIPEDIA..........................................................................................66
FIGURE 11. STUDENTS' REPORTED PREFERENCES FOR ELECTRONIC INFORMATION
RESOURCES................................................................................................................75
FIGURE 12. STUDENTS' REPORTED CONFIDENCE IN THE APPROPRIATENESS OF ELECTRONIC
INFORMATION RESOURCES FOR LEARNING.................................................................78
FIGURE 13. PUPIL DIAMETER FLUCTUATIONS AMONG PARTICIPANTS USING UPTODATE....81
FIGURE 14. PUPIL DIAMETER FLUCTUATIONS AMONG PARTICIPANTS USING ACCESS
MEDICINE...................................................................................................................82
FIGURE 15. PUPIL DIAMETER FLUCTUATIONS AMONG PARTICIPANTS USING WIKIPEDIA.....82
FIGURE 16. EXAMPLE OF A LONGER FIXATION DURATION..................................................83
FIGURE 17. DISTRIBUTION OF KNOWLEDGE TEST SCORES..................................................85
FIGURE 18. GROUP 2 PREFERENCES FOR ELECTRONIC INFORMATION RESOURCES FOR
LEARNING..................................................................................................................88

  v
FIGURE 19. TYPES OF HYPERLINKS IN UPTODATE..............................................................90
FIGURE 20. EXAMPLE OF A QUICK READ FROM WIKIPEDIA.................................................92
FIGURE 21. EXAMPLE OF VIEWING IMAGE IN UPTODATE...................................................94
FIGURE 22. LAYOUT OF MEDIA IN ACCESS MEDICINE.........................................................95
FIGURE 23. LAYOUT OF MEDIA IN WIKIPEDIA.....................................................................96
FIGURE 24. SEARCH PAGE IN UPTODATE...........................................................................97
FIGURE 25. SEARCH PAGE IN ACCESS MEDICINE.................................................................98
FIGURE 26. EXAMPLE OF VIEWING SEARCH PAGE IN UPTODATE........................................99
FIGURE 27. EXAMPLE OF VIEWING SEARCH PAGE IN ACCESS MEDICINE...........................100
FIGURE 28. LEFT-SIDE NAVIGATION MENU IN UPTODATE................................................101
FIGURE 29. EXAMPLE OF STUDENT INTERACTIONS WITH NAVIGATION IN ACCESS MEDICINE
.................................................................................................................................102
FIGURE 30. EXAMPLE OF STUDENT INTERACTIONS WITH NAVIGATION IN WIKIPEDIA.......103
FIGURE 31. EXAMPLE OF READING THE SUMMARY IN UPTODATE....................................108
FIGURE 32. DROP-DOWN SECTIONS IN ACCESS MEDICINE.................................................111

  vi
LIST OF TABLES
TABLE 1. EXAMPLES OF RESEARCH ON THE APPROPRIATENESS OF UPTODATE, ACCESS
MEDICINE, AND WIKIPEDIA.........................................................................................4
TABLE 2. INITIAL REVIEWER AGREEMENT ON THE KNOWLEDGE TEST ANSWERS.............50
TABLE 3. PILOT TEST FEEDBACK.......................................................................................51
TABLE 4. EXAMPLE OF 3 X 3 LATIN SQUARE......................................................................57
TABLE 5. SIX LATIN SQUARES USED IN THE EXPERIMENT.................................................59
TABLE 6. THREE LATIN SQUARES USED WITH GROUP 2....................................................71
TABLE 7. REPORTED PURPOSE, FREQUENCY, AND REASONS FOR USING ELECTRONIC
INFORMATION RESOURCES........................................................................................75
TABLE 8. NORMALITY TEST OF EYE METRICS ASSOCIATED WITH MENTAL EFFORT.........79
TABLE 9. SIGNIFICANCE OF EYE METRICS ASSOCIATED WITH MENTAL EFFORT..............80
TABLE 10. NORMALITY TEST OF THE KNOWLEDGE TEST VARIABLE.................................84
TABLE 11. REPORTED LEVELS OF FAMILIARITY AND PRIOR KNOWLEDGE OF THE DISEASES
...................................................................................................................................86
TABLE 12. AVERAGE NUMBER OF VISITED UNIQUE PAGES...............................................87

  vii
EVALUATING THE APPROPRIATENESS OF ELECTRONIC INFORMATION
RESOURCES FOR LEARNING IN PRE-CLERKSHIP MEDICAL EDUCATION:
AN EYE TRACKING STUDY

Dinara Saparova
Dissertation Supervisor – Dr. Joi L. Moore

ABSTRACT
Current U.S. medical students have begun to rely on electronic information
repositories – such as UpToDate, Access Medicine, and Wikipedia – for their pre-
clerkship medical education. However, it is unclear whether these resources are
appropriate for this level of learning due to factors involving information quality, level of
evidence, and the requisite knowledge base. This study evaluated the appropriateness of
electronic information resources from a novel perspective: the amount of mental effort
learners invest in interactions with these resources and the effects of the experienced
mental effort on learning.
Eighteen first-year medical students read about three unstudied diseases in the
three above-mentioned resources (a total of 54 observations). Their eye movement
characteristics (i.e., fixation duration, fixation count, visit duration, and task-evoked
pupillary response) were recorded and used as psychophysiological indicators of the
experienced mental effort. Post reading, students’ learning was assessed with a multiple-
choice test. Eye metrics and test results constituted quantitative data that were analyzed
according to the repeated Latin square design. Students’ perceptions and observations of

  viii