The University of Toledo The University of Toledo Digital Repository Theses and Dissertations 2013 A study of student engagement with media in online training Christian B. Rogers The University of Toledo Follow this and additional works at: http://utdr.edu/theses-dissertations Recommended Citation Rogers, Christian B., "A study of student engagement with media in online training" (2013). Theses and Dissertations. This Dissertation is brought to you for free and open access by The University of Toledo Digital Repository.
It has been accepted for inclusion in Theses and Dissertations by an authorized administrator of The University of Toledo Digital Repository. For more information, please see the repository's About page. A Dissertation entitled A Study of Student Engagement with Media in Online Training by Christian B. Rogers Submitted to the Graduate Faculty as partial fulfillment of the requirements for The Doctor of Philosophy Degree in Curriculum & Instruction _________________________________________ Judy Lambert, Ph., Committee Chair _________________________________________ Tony Sanchez, Ph., Committee Member _________________________________________ Gregory E., Committee Member _________________________________________ Savilla Banister, Ph., Committee Member _________________________________________ Dr.
Patricia Komuniecki, Ph., Dean, College of Graduate Studies The University of Toledo May 2013 Copyright © 2013, Christian B. Rogers This document is copyrighted material. Under copyright law, no parts of this document may be reproduced without the expressed permission of the author. An Abstract of A Study of Student Engagement with Media in Online Training Christian B.
Rogers Submitted to the Graduate Faculty as partial fulfillment of the requirements for The Doctor of Philosophy in Curriculum and Instruction The University of Toledo May 2013 Many studies have been conducted where a face-to-face training environment is compared to an online training environment. While some research has been conducted on the nature of online training in faith-based not-for-profit organizations, little to no research has been found on engagement. The purpose of this study was to determine if the level of engagement of participants in a training course for new staff and interns with Campus Crusade for Christ would be increased by conducting training online instead of face-to-face and by utilizing multiple forms of media. The survey that was utilized included questions adapted from the Student Course Engagement Questionnaire (Handelsman et al., 2005) and was analyzed utilizing the Rasch measurement model to understand whether the survey successfully met the requirements for measuring engagement.
The Rasch measurement analysis revealed that the survey was weak and did not measure engagement, thus the results of the survey revealed no significant differences in the level of engagement. Further research is recommended with new questions being added to the survey that are considered to have a greater level of difficulty as well as research should be conducted that involve qualitative data collection. iii This dissertation is dedicated to the memory of my grandmother, Jennie Rogers. She believed in me from a young age that I could do anything that I desire and she would be proud of me in whatever path God takes me.
Acknowledgements This dissertation would not be possible without the significant contribution of others. My gratitude is beyond measure for their help and dedication. Many thanks to my dissertation committee: To Dr. Judy Lambert, who believed in the cause of my dissertation and worked with me on not only this study but my own development as a researcher and writer.
She took countless hours of her time to review my writing and gave suggestions that helped hone my study and give it direction. Stone, who made himself available on multiple occasions for the review of my methodology and for his coaching through the Rasch measurement process. Stone’s expertise in measurement was invaluable to this study. Tony Sanchez, who provided a new perspective in reviewing of my writing with his many years of academic expertise and his willingness to sit with me and discuss multiple facets of my study.
Savilla Banister, who served as my coach and friend through this process. I will always remember the extra care and time you took to help me in reviewing my writing and also providing me a healthy perspective on the doctoral process. v Table of Contents Abstract iii Acknowledgments v Table of Contents vi List of Tables x List of Figures xi I. Statement of the Problem 4 B.
Need for the Study 4 C. Purpose and Importance of the Study 4 D. Research Questions 5 II. Training in Organizations 6 a.
Purpose of training 6 b. Theories for the development of training 7 Adult learning theory 7 Self-directed learning theory 9 Constructivist learning theory 10 c. Models for the development of training 11 d. ADDIE 11 Morrison, Ross, and Kemp 12 e.
Distance education training 14 Definition of distance education 14 History of distance education 15 vi Characteristics of distance education 19 Advantages of distance education 23 Disadvantages of distance education 26 f. Evaluation of training 29 Kirkpatrick’s four levels 29 Course experience questionnaire 31 B. Faith-Based Organizations 32 a. Characteristics of faith-based organizations 32 b.
Training in faith-based organizations 33 c. Campus Crusade for Christ 35 C. Engagement During Training 35 a. Media that promotes engagement 37 Multimedia 38 Tools for interaction 39 Tools for presentation 40 c.
Measurement of engagement 41 D. Scope of the Study 44 B. Data Collection and Instrumentation 45 C. Overview of Rasch measurement 47 vii Rasch introduction 48 Concepts underpinning the Rasch model 48 Fit 50 D.
Limitations of the Study 50 IV. Rating scale analysis 51 b. Dimensionality analysis 53 Baseline statistics 53 Principle contrast analysis 56 c. Person-Item map 62 B.
Analysis of Variance 68 C. Findings 72 Rating scale analysis 73 Summary statistics 73 Dimensionality 76 B. Research Question 2 80 Unclear expectations 81 Lack of media 81 Lack of collaboration 81 D. Items in training 82 b.
Qualitative Data 84 References 87 Appendix A: Student Engagement Survey 95 ix List of Tables Table 1. Rating Scale Threshold Structure. Summary Statistics for Module 1 for 66 Persons. Summary Statistics for Module 1 for 66 Items.
Summary Statistics for Module 2 for 66 Persons. Summary Statistics for Module 2 for 12 Items. Summary Statistics for Module 3 for 66 Persons. Summary Statistics for Module 3 for 12 Items.
Summary Statistics for Module 4 for 66 Persons. Summary Statistics for Module 4 for 12 Items. Summary Statistics for Module 5 for 66 Persons. Summary Statistics for Module 5 for 12 Items.
Explained and Unexplained Variance. Factors of Engagement. One-Way Analysis of Variance for Module 1. One-Way Analysis of Variance for Module 2.
One-Way Analysis of Variance for Module 3. One-Way Analysis of Variance for Module 4. One-Way Analysis of Variance for Module 5. Factors of Survey Questions.
Survey Questions in Relation to Mean. Overview of Modules. Question Suggestions for Survey. 84 xi List of Figures Figure 1.
Components of the Morrison, Ross, and Kemp Design Model. Equation for Rasch Measurement. Category Probabilities for Questions 1 – 9. Category Probabilities for Questions 10 – 12.
Variable Map for Module 1. Variable Map for Module 2. Variable Map for Module 3. Variable Map for Module 4.
Variable Map for Module 5. 67 xii \Chapter One Introduction Not-for-profit organizations are similar in many ways to for-profit organizations. The way the organization is structured, how they obtain funding, and the type of employee they recruit may differ but many of the operations are similar. Similarities may include organizational structure, expectations of employees to understand how that structure works, payroll, and human resources.
In order to sustain the health of a not-for-profit organization, training is often conducted for new staff, existing staff, and interns. One type of organization within the not-for-profit sector is that of a faith-based organization. A faith-based organization may be a church, missions’ agency, or even social justice organization such as the Salvation Army. Just like other not-for-profit organizations, faith-based organizations need to train new staff at an initial and ongoing basis.
The purpose of training a person within a faith- based organization may include how to educate someone on his or her role within the organization, how the organizational structure functions, or the skills needed for conducting the job. Other foci might be how to counsel specific types of people, how to relate to others that are within the organization, or ways to grow personally and spiritually. Faith-based organizations deal with certain obstacles when they develop training for staff and interns. One of the more prominent obstacles is the engagement of participants.
If a course does not engage participants, learning may not take place. The way that the materials are presented can have a major effect on participants’ level of engagement. Moreover, the environment that is chosen for the training can dictate the 1 way that materials are presented. For example, in a face-to-face training format, materials may be presented via printed documents.
In an online format or a blended format (a combination of face-to-face and online), course materials may be presented using text-based documents, group discussions using online discussion boards, videos, or audio. Campus Crusade for Christ (Cru), a faith-based organization, considered which training environment would result in new staff and interns being more engaged with the course materials. Cru recruits over 600 new staff and interns each year. Training is required for interns and full-time staff members.
Currently, Cru staff and interns are trained at specific locations where they are assigned. Upon arriving at that location, each trainee is paired with a coach with whom they meet on a weekly basis for two years. During each of these training sessions, a coach will teach the materials and engage with the trainee in hands-on experiences, as well as discuss readings. Trainees receive these materials by downloading Portable Document Format (PDF) files from a wiki and printing them to read in preparation for meeting with the coach.
Wikis are online collaborative editing tools that allow participants to become editors of a topic at any time and from any location (Leuf & Cunningham, 2001). The current curriculum that Cru has developed is based on what is called the five E’s of development that include environment, education, exposure, experience, and evaluation. Environment is the culture of growth that Cru seeks to create for each intern and new staff member. Education is the opportunity that Cru wants to create for each new staff member and intern through the use of the training as well as other venues such as workshops and retreats.
Exposure is provided through extended learning opportunities 2 such as by offering curriculum by experts in particular fields and mentoring experiences for new staff and interns by senior staff members. Experience is when new staff and interns practice their newly-learned skills. Experiential activities, which are offered within each module, might include having a discussion with five students on a specific topic. Evaluation is when new staff and interns receive constructive feedback within each module.
By utilizing the five E’s, Cru promotes a culture of education and self- study, along with skill development on spiritual leadership, personal counseling of students, organizational structure, and procedures. According to Cru’s national training director, participants of the face-to-face training environment demonstrated a lack of engagement with the materials in the course. With the aim of increasing participant engagement, the director developed a distance education course that provided participants with training through a variety of online media and materials. The study examined whether the online environment and related media increased participant engagement and was limited to training that is offered to staff and interns in Cru’s Great Lakes region, which includes Ohio, Michigan, Indiana and Illinois.
The online environment was asynchronous, meaning that participants had the flexibility to access course materials when it was convenient and go through each course module at their own pace with the same deadlines for ending each module as the face-to- face training participants.