Andrews University Digital Commons @ Andrews University Dissertations Graduate Research 2018 Student Evaluations of Teaching, Course and Student Characteristics at Andrews University Fatimah Al Nasser Andrews University, fatimah@andrews.edu Follow this and additional works at: https://digitalcommons.edu/dissertations Part of the Higher Education and Teaching Commons Recommended Citation Al Nasser, Fatimah, "Student Evaluations of Teaching, Course and Student Characteristics at Andrews University" (2018).edu/dissertations/1662 This Dissertation is brought to you for free and open access by the Graduate Research at Digital Commons @ Andrews University. It has been accepted for inclusion in Dissertations by an authorized administrator of Digital Commons @ Andrews University. For more information, please contact repository@andrews. ABSTRACT STUDENT EVALUATIONS OF TEACHING, COURSE AND STUDENT CHARACTERISTICS AT ANDREWS UNIVERSITY by Fatimah Al Nasser Chair: Larry D.
Burton ABSTRACT OF GRADUATE STUDENT RESEARCH Dissertation Andrews University School of Education Title: STUDENT EVALUATIONS OF TEACHING, COURSE AND STUDENT CHARACTERISTICS AT ANDREWS UNIVERSITY Name of researcher: Fatimah Al Nasser Name and degree of faculty chair: Larry D. Date completed: July 2018 Problem This research examined the type of rating of specific Student Evaluation of Teaching (SET) dimensions and overall rating students tended to give for the courses that they took, identified the dimensions that predicted the overall rating, and assessed the association of gender, student academic status, course level, course type, academic school and the effect on SET scores. Method The researcher used a quantitative research method to explore the type of score that students give to the courses they took, examine the relationship between dimensions of SET and overall rating, and the influence of gender, student status, course level, course type, and academic school in SET score. The study included 3,745 responses to courses at five schools at Andrews University.
Andrews University’s Course Survey was used as the main instrument. Descriptive analysis, regression linear analysis, and multivariate analysis of variance were conducted to help answer the research questions. Results The research found that students tended to rate all courses highly. However, these students tended to rate the dimensions of respect for diversity, preparation and organization, and availability and helpfulness higher than other dimensions.
Four dimensions were found predicting SET overall rating. These dimensions were: stimulate interest, effective communication, intellectual discovery and inquiry, and evaluation and grading. Regarding the student and course characteristics that were examined, gender was not found to influence the SET score. Student academic status and course level were found to affect SET scores within specific dimensions, but the effect sizes were very small.
Both the course type and the academic school were found not significant enough to be used in practice. Conclusion This study supported other research that reported some dimensions of SET predicted overall rating. The research offered a model with four dimensions that predicted overall rating. The results of this study supported the theory that both student status and course level affect SET scores.
However, this study found that the effect of these two factors tended to be within specific dimensions of SET. Different from other studies, this study found that gender had no influence on SET scores. Both course type and academic school had a very small effect size, which is not large enough to be used in practice Andrews University School of Education STUDENT EVALUATIONS OF TEACHING, COURSE AND STUDENT CHARACTERISTICS AT ANDREWS UNIVERSITY A Dissertation Present in the Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy by Fatimah Al Nasser July 2018 © Copyright by Fatimah Al Nasser 2018 All Rights Reserved STUDENT EVALUATIONS OF TEACHING, COURSE AND STUDENT CHARACTERISTICS AT ANDREWS UNIVERSITY A dissertation presented in partial fulfillment of the requirements for the degree Doctor of Philosophy by Fatimah Al Nasser APPROVAL BY THE COMMITTEE: ____________________________________________ _______________________________________ Chair: Larry D. Burton Dean, School of Education Robson Marinho ________________________________ Member: Jimmy Kijai ________________________________ Member: Lynn Merklin ________________________________ ______________________________ External: Alice C Williams Date approved TABLE OF CONTENTS LIST OF TABLES.
vi LIST OF ABBREVIATIONS. 1 Background of the Problem. 4 Statement of the Problem. 4 Purpose of the Study.
6 Student Evaluations of Teaching. 6 SET Overall Rating and SET Dimensions. 7 Student and Course Characteristics Influencing SET Scores. 9 The Research Questions.
11 Significance of the Study. Erro r! Bookmark not defined. 12 Definition of Terms. 14 Student Evaluation of Teaching.
15 Introduction to SET. 18 Challenges in SET. 20 Validity and Reliability of SET. 21 SET and Bias Factors.
23 Dimensions of Effective Teaching. 24 iii Dimensions of SET and SET Overall Rating. 26 Student and Course Characteristics Affecting SET. 29 Summary of Literature Review.
48 Need for Further Study. 55 The Research Questions. 59 SET Overall Rating. 65 Characteristics of Participants.
66 Results by Research Questions. 76 Gender and SET Score. 78 Student Status and SET Score. 78 Course Type and SET Score.
84 Course Level and SET Score. 87 Academic School and SET Score. 88 Summary of Major Findings. SUMMARY, DISCUSSIONS, CONCLUSIONS, AND RECOMMENDATIONS.
99 Review of Literature and Conceptual Framework. 99 Review of Literature. 109 Purpose of the Study. 110 iv Research Design.
112 Summary of Findings. 112 Findings for Research Question 1. 113 Findings for Research Question 2. 113 Findings for Research Question 3.
ANDREWS UNIVERSITY SET INSTRUMENT. TABLE OF DEFINITON OF VARIABLES. IRB FORM AND CERTIFICATION. 148 v LIST OF TABLES 1.
Summary of Studies Compared to the SET Dimensions and the Examined Characteristics in this Study. Schools and the Number of Responses for Each One. SET Dimensions (Variables), Conceptual Definition, Items, Source. Characteristics of Responses.
Descriptive Analysis of SET Dimensions Related to the Course and Instructor. Descriptive Analysis of Questions Related to Overall Rating. Correlations Between Overall Rating and SET Dimensions. Standard Regression Analysis Result (Full Model) for the Predictors for Overall Rating.
Standard Regression Analysis Result (Restricted Model with Major Predictors). Standardized Coefficients in Two Models and the Value of R2. Mean and Standard Deviation for Gender Group. The Mean, and Standard Deviation for the Student Status Groups.
Between-Subject (Student Status) Effects. Mean Differences Between Student Status. The Mean and Standard Deviation for the Course Type Groups. Between-Subjects (Course Type) Effects.
The Mean and Standard Deviation for the Course Level Groups. Between-Subjects (Course Level) Effects. Mean Differences Between Course Level. The Mean and Standard Deviation for the Academic School Groups.
Between-Subjects (Academic School) Effects. Research Questions and Dimensions Found Related. 98 vii LIST OF ABBREVIATIONS ANOVA Analysis of Variance MANOVA Multivariate Analysis of Variance SET Student Evaluation of Teaching SPSS Statistical Package for the Social Sciences VIF Variance Inflation Factor viii ACKNOWLEDGEMENT In the name of God, the Most Gracious, the Most Merciful I first thank Allah, my Creator and Master, the great teacher and messenger prophet Mohammad, the Immaculate Imams, Imam Hussain, and Imam Mahdi. I thank my parents, for always being there for me and inspiring me to reach a high level of education.
Thank you for your prayers that made a hard time much easier. I also thank all of my family members who prayed for me to reach my goals. Thank you to my sister and brother who always believed that I can reach all my dreams. Thank you to my nephew, Hassan, who was always there to help and support during this journey.
I thank my committee chair, Dr. Larry Burton for your guidance, thoughtful feedback, encouragements, and time during the process of completing this work. I would like to acknowledge the support of my committee members Dr. Kijai and Dr.
Thank you Dr. Kijai for reading the documents many times, helping to modify many tables, and enlightening me with your knowledge. Thank you Dr. Merklin for providing thoughtful suggestions for clarity and willing to help me all the time.
I would like to acknowledge the assistance of Laura Carroll and Amy Waller. Thank you Mrs. Carroll for all your help with the data extraction process. Thank you Mrs.
Waller for all your help finding the documents that I needed during the process of writing my dissertation. Thank you to all my friends and colleagues who encouraged me and prayed for me to complete this work with easiness and joy. ix CHAPTER 1 INTRODUCTION Background of the Problem The meaning of effective teaching is expanding faster than before. Effective teaching includes using effective teaching methods, having knowledge, and making students interested in learning (Evans, Baskerville, Wynn-Williams, & Gillett, 2014).
Student Evaluation of Teaching (SET) (McIntyre, Smith, & Hassett, 1984) is an important tool that most higher education institutions use to help measure teaching effectiveness (Hobler, 2014) and as a tool for faculty evaluation systems around the world (Al-Issa & Sulieman, 2007). A review of the literature indicated that SET is an assessment, which takes the form of a survey that is completed by the student at the end of a course or a program. This survey asks the students to use their judgment to report their experiences regarding the effectiveness of the instructor or the quality of the course (Ali & Ajmi, 2013; Brown, 2008; Driscoll & Cadden, 2010; Hobson & Talbot, 2001; Lindahl & Unger, 2010; Oliver & Pookie, 2005; Smith, 2007; Tsai & Lin, 2012). The items that are included in the SET survey are related to overall rating and the dimensions of effective teaching.
Overall rating measured the student general opinion regarding the course, instructor and learning experience. The dimensions of effective teaching related to the principles of effective teaching that an institution adapts. Each higher education institution has its own definition of effective teaching that emphasizes 1 specific dimensions that might not be emphasized by other institutions. Therefore, the SET survey might differ from one institution to another.
Research found that effective teaching should encourage student-faculty contact, cooperation among students, active learning, communicate high expectations, provide prompt feedback, emphasize time on task, and respect diverse talents and ways of learning (Chickering & Gamson, 1989). Other research reported that effective teaching included other aspects such as intellectual growth (Bowman & Seifert, 2011), course content and critical thinking (Anderson, 2012), course structure (Lumpkin & Multon, 2013), communication (Nargundkar & Shrikhande, 2012), respect for diversity, organization, and clarity (Lumpkin & Multon, 2013). The result of this survey is used for different purposes. It is used by administrators and instructors to make important decisions regarding development of the course and the instruction (Al-Issa & Sulieman, 2007; Ali & Ajmi, 2013; Beuth et al.
Also, it is used by students to make better decisions regarding which course they want to take and to be aware of the course levels of difficulty before the registration process (Adams & Umbach, 2012; Ali & Ajmi, 2013; Atek, Salim, Ab Halim, Jusoh, & Yusuf, 2015; Beuth et al., 2015; Brockx, Spooren, & Mortelmans, 2011; Chulkov & Jason Van, 2012; Donnon, Delver, & Beran, 2010; Driscoll & Cadden, 2010; Fah, Yin & Osman, 2011). Furthermore, SET is considered an important tool that could have negative impacts on the development of teaching if it does not give accurate results (Ali & Ajmi, 2013). Additionally, institutional administrators use SET for accreditation purposes and to make decisions for the faculty promotion process (Terry, Heitner, Miller, & Hollis, 2017). Therefore, accurate results help develop better instruction and more knowledgeable learners (Hobler, 2014).
2 Some researchers have suggested that the results of SET are not truly a reflection of effective teaching. They proposed that different non-instructional factors could affect the results of SET or produce biased results, and called for further studies (Coffman, 1954; Reynolds, 1979).