Evaluating the effectiveness of the Master of Science in Agronomy distance program

184 Pages·2017·3.52 MB·English

Iowa State University Capstones, Theses and
Graduate Theses and Dissertations
Dissertations
2015
Evaluating the effectiveness of the Master of
Science in Agronomy distance program: a
comprehensive investigation of learners’
perspective of quality in distance learning
Asimoula Valai
Iowa State University
Follow this and additional works at:https://lib.dr.iastate.edu/etd
Part of theEducational Assessment, Evaluation, and Research Commons,Higher Education
Administration Commons,Higher Education and Teaching Commons, and theInstructional Media
Design Commons
Recommended Citation
Valai, Asimoula, "Evaluating the effectiveness of the Master of Science in Agronomy distance program: a comprehensive investigation
of learners’ perspective of quality in distance learning" (2015).Graduate Theses and Dissertations. 14864.
https://lib.dr.iastate.edu/etd/14864
This Thesis is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital
Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital
Repository. For more information, please [email protected].
Evaluating the effectiveness of the Master of Science in Agronomy distance program: A
comprehensive investigation of learners’ perspective of quality in distance learning

by

Asimoula Valai

A thesis submitted to the graduate faculty

in partial fulfillment of the requirements for the degree of

MASTER OF SCIENCE

Major: Education (Curriculum and Instructional Technology)

Program of Study Committee:
Denise Schmidt-Crawford, Major Professor
Constance Hargrave
Kenneth Moore

Iowa State University

Ames, Iowa

2015

Copyright © Asimoula Valai, 2015. All rights reserved.
ii

DEDICATION

This thesis work is dedicated to my husband, Sotirios, who has been a source of love
and encouragement during the challenges of graduate school and family life and to my son
Vaios, the biggest joy in my life. I am truly thankful for having you both in my life. This
work is also dedicated to my parents, Georgios and Stamatia Valai, who have always loved
me unconditionally and whose good examples have taught me to work hard with patience
and love for the things that I aspire to achieve.
iii

TABLE OF CONTENTS

Page
LIST OF FIGURES ................................................................................................... vi
LIST OF TABLES ..................................................................................................... vii
ACKNOWLEDGMENTS ......................................................................................... viii
ABSTRACT ............................................................................................................... x
CHAPTER 1 GENERAL INTRODUCTION ....................................................... 1
Introduction ......................................................................................................... 1
Thesis Organization ............................................................................................. 4
References ......................................................................................................... 5

CHAPTER 2 QUALITY INDICATORS FOR DISTANCE LEARNING: A
LITERATURE REVIEW IN LEARNERS’ PERCEPTIONS ................................... 8
Abstract ......................................................................................................... 8
Introduction ......................................................................................................... 8
Defining quality and quality indicators.......................................................... 10
Defining distance learning ............................................................................. 11
Common themes of quality standards in distance education ......................... 12
Characteristics of distance earners: The profile ............................................. 15
Scope of the Research .......................................................................................... 16
Results and Discussion ........................................................................................ 17
Course design ................................................................................................. 17
Interaction ...................................................................................................... 22
Support ..................................................................................................... 25
Technology use .............................................................................................. 27
Conclusion ......................................................................................................... 30
References ......................................................................................................... 32
CHAPTER 3 EVALUATING DISTANCE LEARNING: QUALITY
INDICATORS FROM A LEARNERS’ PERSPECTIVE ......................................... 41
Abstract ......................................................................................................... 41
Introduction ......................................................................................................... 42
The role of evaluation in the Master of Science in Agronomy distance
program ........................................................................................................ 43
Theoretical Frameworks ...................................................................................... 44
Logic model for program evaluation ............................................................. 44
Dimensions and indicators of quality ............................................................ 47
Methodology ........................................................................................................ 49
iv

Participants .................................................................................................... 50
Data Collection ............................................................................................. 50
Data Analysis ................................................................................................. 51
Findings and Discussion ..................................................................................... 51
Learning outcomes ........................................................................................ 52
Courses ........................................................................................................ 53
Interaction ...................................................................................................... 56
Use of technology .......................................................................................... 58
Support ........................................................................................................ 60
Conclusion ........................................................................................................ 62
References ......................................................................................................... 63

CHAPTER 4 A LONGITUDINAL PROGRAM EVALUATION IN
LEARNERS’ PERCEPTIONS OF QUALITY IN DISTANCE LEARNING .......... 67
Abstract ......................................................................................................... 67
Introduction ......................................................................................................... 68
Purpose of the evaluation ............................................................................... 70
Evaluation stakeholders ................................................................................. 70
Evaluation key questions ............................................................................... 70
Theoretical Perspective ........................................................................................ 71
Logic model for program evaluation ............................................................. 71
Underpinning educational theories applied in distance learning ................... 74
Effective learning environments .................................................................... 76
Methods ......................................................................................................... 78
Study design ................................................................................................... 79
Participants ..................................................................................................... 80
Quantitative phase .......................................................................................... 82
Qualitative phase ............................................................................................ 83
Results ......................................................................................................... 85
Quantitative phase .......................................................................................... 85
Quality indicators .................................................................................. 85
Reliability of indicators......................................................................... 87
Standards ............................................................................................... 88
Quality of the program .......................................................................... 90
Changes over time................................................................................. 106
Qualitative phase ............................................................................................ 115
Epoche................................................................................................... 116
Significant statements ........................................................................... 117
Themes .................................................................................................. 118
Discussion ......................................................................................................... 125
Learner-centered effectiveness of the program .............................................. 127
Knowledge-centered effectiveness of the program ........................................ 127
Assessment-centered effectiveness of the program ....................................... 128
Community-centered effectiveness of the program ....................................... 130
Conclusion ......................................................................................................... 131
References ....................................................................................................... 133
v

CHAPTER 5 GENERAL CONCLUSIONS ......................................................... 138
APPENDIX A HUMAN SUBJECT REVIEW COMMITTEE APPROVAL
FOR OUTCOME EVALUATION ............................................................................ 142
APPENDIX B HUMAN SUBJECT REVIEW COMMITTEE APPROVAL
FOR LONGITUDINAL EVALUATION.................................................................. 143
APPENDIX C SUMMARY STATISTICS OF ALL QUALITY
INDICATORS’ AVERAGE BY YEAR ................................................................... 144
APPENDIX D SUMMARY STATISTICS OF QUALITY INDICATORS .......... 145
APPENDIX E INDICATORS’ AVERAGE BY YEAR ....................................... 146
APPENDIX F INTERCEPT AND SLOPE OF SCORES IN EFFECTIVE
LEARNING ENVIRONMENTS OVER TIME ....................................................... 147
APPENDIX G SIGNIFICANT STATEMENTS .................................................... 148
APPENDIX H M.S. IN AGRONOMY EDUCATION PROGRAM EXIT
SURVEY ............................................................................................................ 154
APPENDIX I M.S. IN AGRONOMY DISTANCE EDUCATION
PROGRAM EXIT INTERVIEW PROTOCOL ........................................................ 174
vi

LIST OF FIGURES

Page

Figure 3.1 M.S. in Agronomy Distance Education program Logic model,

adapted from Kellogg Foundation’s (2004) Logic Model Development Guide ....... 45

Figure 3.2 Analysis of Means (ANOM) displaying a significant difference

between Presence and Importance of indicator 16 .................................................... 57

Figure 4.1 M.S. in Agronomy Distance Education program Logic model,

adapted from Kellogg Foundation’s (2004) Logic Model Development Guide ....... 72

Figure 4.2 Visual model for mixed methods explanatory design procedures ............ 81

Figure 4.3 Distribution graphs of learner-centered Indicators ................................... 92

Figure 4.4 Distribution graphs of knowledge-centered Indicators ............................ 94

Figure 4.5 Distribution graphs of assessment-centered Indicators ............................ 101

Figure 4.6 Distribution graphs of community-centered indicators ............................ 103

Figure 4.7 Analysis of quality indicators means (ANOM) ........................................ 106

Figure 4.8 Line of best fit for scores over time.......................................................... 107

Figure 4.9 Score values of years scored higher by indicator ..................................... 109

Figure 4.10 Score values of years scored lower by indicator .................................... 110

Figure 4.11 Changes over time for learner-centered indicators ................................ 112

Figure 4.12 Changes over time for knowledge-centered indicators .......................... 113

Figure 4.13 Changes over time for assessment-centered indicators .......................... 114

Figure 4.14 Changes over time for community-centered indicators .......................... 115
vii

LIST OF TABLES
Page

Table 2.1 Common themes of quality standards in distance learning ....................... 14
Table 2.2 Studies in learners’ perspectives of quality in distance learning ............... 18

Table 2.3 Learners’ perceptions of quality in distance learning identified in the

literature .................................................................................................................... 31
Table 3.1 Overall mean scores of dimensions & differences between the criteria

in each dimension ...................................................................................................... 52

Table 3.2 Quality indicators and scores related to program Learning outcomes ....... 54
Table 3.3 Quality indicators and scores related to Courses ....................................... 56
Table 3.4 Quality indicators and scores related to Interaction ................................... 59
Table 3.5 Quality indicators and scores related to Use of technology ....................... 60
Table 3.6 Quality indicators and scores related to Support ....................................... 61
Table 4.1 Post-survey indicators of program’s effectiveness .................................... 86

Table 4.2 Scales of effective learning environments ................................................. 87
Table 4.3 Cronbach’s α among scales of effective learning environments ............... 88
Table 4.4 Scales and Indicators in quantitative analysis ............................................ 89
Table 4.5 Standards for quality indicators ................................................................. 90
Table 4.6 Analysis of Means summary for learner-centered quality indicators ........ 93
Table 4.7 Analysis of Means summary for knowledge-centered indicators .............. 100
Table 4.8 Analysis of Means summary for assessment-centered indicators ............. 102
Table 4.9 Analysis of Means summary for community-centered indicators ............. 104
viii

Table 4.10 Analysis of years’ means summary ......................................................... 108
Table 4.11 Theme of learner-centered effectiveness and evidence ........................... 120
Table 4.12 Theme of knowledge-centered effectiveness and evidence ..................... 122
Table 4.13 Theme of assessment-centered effectiveness and evidence .................... 124
Table 4.14 Theme of community-centered experiences and evidence ...................... 126
ix

ACKNOWLEDGMENTS

I would first like to acknowledge the guidance of my major professor, Dr. Denise
Schmidt-Crawford. She has been instrumental in encouraging me in order to grow and
succeed during my graduate school career.
Secondly, I would like to thank my committee member, Kenneth Moore and the
Department of Agronomy at Iowa State University for providing me the opportunity to
conduct my research in the Master of Science in Agronomy distance program. I also want to
acknowledge my committee members Dr. Constance Hargrave and Dr. Larysa Nadolny, for
their suggestions and support.
In addition, I would also like to thank my friends, colleagues, the department faculty
and staff for making my time at Iowa State University a wonderful experience. I want to also
offer my appreciation to those who were willing to participate in my surveys and interviews,
without whom, this thesis would not have been possible.