Louisiana State University LSU Digital Commons LSU Doctoral Dissertations Graduate School 2005 Toward a construct validation of the Louisiana School Analysis Model Instructional Staff Questionnaire Nikki Bray Clark Louisiana State University and Agricultural and Mechanical College, nclark4@lsu.edu Follow this and additional works at: https://digitalcommons.edu/gradschool_dissertations Part of the Human Resources Management Commons Recommended Citation Clark, Nikki Bray, "Toward a construct validation of the Louisiana School Analysis Model Instructional Staff Questionnaire" (2005). LSU Doctoral Dissertations.edu/gradschool_dissertations/3602 This Dissertation is brought to you for free and open access by the Graduate School at LSU Digital Commons. It has been accepted for inclusion in LSU Doctoral Dissertations by an authorized graduate school editor of LSU Digital Commons. For more information, please contactgradetd@lsu.
TOWARD A CONSTRUCT VALIDATION OF THE LOUISIANA SCHOOL ANALYSIS MODEL INSTRUCTIONAL STAFF QUESTIONNAIRE A Dissertation Submitted to the Graduate Faculty of the Louisiana State University and Agricultural and Mechanical College in partial fulfillment of the requirements for the degree of Doctor of Philosophy in The School of Human Resource Education and Workforce Development by Nikki Bray Clark B., Southeastern Louisiana University, 1986 M., Southeastern Louisiana University, 1996 May, 2005 ©Copyright 2005 Nikki Bray Clark All rights reserved. ii ACKNOWLEDGEMENTS This study would not have been possible without the support and cooperation of many people. I will always be grateful to my major professor and the chair of my doctoral committee Dr. Reid Bates, for challenging me to grow and providing the guidance and assistance necessary to make this a better study.
I extend my sincere appreciation to the members of my doctoral committee for their support and suggestions in this endeavor: Dr. Michael Burnett, Dr. Geraldine Johnson, and Dr. I am thankful for the encouragement and numerous discussions with my colleague and dear friend, Dr.
The love and support of my parents has given me the solid foundation that has made my lifelong pursuit of learning possible. Most importantly, I am deeply grateful to my husband, Joey, whose love, understanding, faith in my abilities, and unwavering support were my solace and refuge. And lastly, a heartfelt thanks to my son Scott who never complained about his mother always being in school. iii TABLE OF CONTENTS ACKNOWLEDGMENTS.
ii LIST OF TABLES. vi LIST OF FIGURES. vii LIST OF ACRONYMS. ix CHAPTER ONE INTRODUCTION .1 Historical Context of Education in Louisiana .2 French Colonial Period .2 Spanish Colonial Period .4 Civil War and Reconstruction .5 Early 20th Century Louisiana .6 Modern Education in Louisiana .7 Continued Need for Improvement .8 Evolution of Standards and Assessments .8 Grade Level Expectations .9 Accountability Legislative Mandate .10 Louisiana’s Original Accountability System .12 Louisiana’s New Federally Approved Accountability System.13 School Analysis Model .13 The SAM Instructional Staff Questionnaire .19 CHAPTER TWO REVIEW OF LITERATURE .20 Criticism and Reform .20 Federal Intervention in Education .22 No Child Left Behind .23 Change and School Effectiveness .25 Change and Organizational Development .26 Change and Moral Purpose.29 Social Systems Theory .30 iv Social Cognitive Theory .33 Accountability and Student Achievement .33 Accountability and Public Policy.34 Accountability and Diversity .37 Evolution of School Effectiveness Research .38 Criticism of School Effectiveness Research .40 Key Components of School Effectiveness .41 Expectations of Teachers .44 Alternative Certification and Licensure .44 Successful Teaching Experience .45 Self-Efficacy and Teacher Commitment .45 Self-Efficacy and Stress Management .46 Self-Efficacy and Professional Development Effectiveness .47 Self-Efficacy and Teaching Effectiveness .51 Poverty and School Effectiveness .56 Poverty and Student Achievement .56 Poverty and High Performance .58 Poverty and Equity .59 Poverty and Resources .59 Poverty and School/District Size .61 Poverty and Parental Involvement .62 Poverty an Site Autonomy .63 CHAPTER THREE RESEARCH METHODOLOGY .64 Research Design Overview.67 Conceptual/Operational Definitions .74 Additional Independent Variables .75 Data Analysis Procedures .76 Analysis: Research Question One.76 Research Question One.76 Exploratory Factor Analysis .77 Factor Analysis Decision Rules .78 Sample Size Requirements .79 Analysis: Research Question Two .79 Research Question Two .79 Analysis: Research Question Three .80 Research Question Three .80 Hierarchical Multiple Regression .81 Multicolinearity Assessment Rules.81 CHAPTER FOUR QUANTITATIVE RESULTS .83 Results of the Factor Analyses.88 Eleven and Ten Factor Solution.88 Nine Factor Solution .89 Eight Factor Solution .90 Seven Factor Solution .91 Results of the Correlation Analyses.96 School Performance Scores .97 Size of School .97 Teacher Participation in School Decision Making .97 Teachers Perceptions of Student Ability .97 Parental Concern About Child Achievement.99 School Teaching Effectiveness .99 Effectiveness of Staff Development Activities.99 Summary of Correlation Analyses.99 Results of the Regression Analyses .103 Hierarchical Multiple Regression .106 CHAPTER FIVE FINDINGS, CONCLUSIONS AND IMPLICATIONS .108 Restatement of the Research Problem .108 Summary of the Methodology .109 Summary of Findings.110 Comparison of the SISQ with the Originally Proposed Structure .111 Improving the SISQ .115 Teacher Participation in School Decision Making .116 Teacher Participation in School Improvement Activities .117 Teacher Perceptions of Student Ability .120 Parental Concerns about Child Achievement .123 School Teaching Effectiveness .124 Dimension A: Student Safety .125 Dimension B: High Staff Morale .125 Dimension C: Staff Interaction .126 Dimension D: Student Staff Interaction .126 Staff Development Effectiveness.127 School Teaching Effectiveness .128 Teacher Self Efficacy .129 Summary of the SISQ and its Improvement Needs .129 Additional Construct Validation Steps .131 School Performance Score.134 Teacher Participation in School Decision Making .134 Hierarchical Regression Analysis .136 General Implications and Recommendations .173 viii LIST OF TABLES 1.
1998-1999 School Performance Category Assignment. SAM Instructional Staff Questionnaire Item/Category System Controls. SAM Instructional Staff Questionnaire Item/Category Climate. SAM Instructional Staff Questionnaire Item/Category Culture.
SAM Instructional Staff Questionnaire Item/Category Parent and School Relations. SAM Instructional Staff Questionnaire Item/Category Curriculum and Instruction. SAM Instructional Staff Questionnaire Item/Category Staff Development. SAM Instructional Staff Questionnaire Item/Category Administrator Leadership.
Profile of Teacher Sample –Louisiana Middle School Teachers Teaching Experience. Profile of Teacher Sample – Louisiana Middle School Teachers Teaching Experience at Present School. Profile of Teacher Sample – Louisiana Middle School Teachers Highest Education Level Attained. Profile of Teacher Sample – Louisiana Middle School Teachers Days Absent for Professional Development.
Factor Loadings for the Eight-Factor Oblique Solution for the SAM School Analysis Instructional Staff Questionnaire (SISQ) .6 Descriptive Statistics for the Instructional Staff Questionnaire (SISQ) SAM Instructional Staff Questionnaire Eight-Factor Solution.7 Instructional Staff Questionnaire (SISQ) SAM Instructional Staff Questionnaire Item Distribution for the Eight-Factor Solution. Summary of the Pearson Correlation Coefficients for the SAM School Analysis Staff Questionnaire (SISQ) Scales and the Control Variables. Summary of Hierarchical Multiple Regression of the School Performance Score (SPS) on Independent Control Variables for Model 1. Summary of Hierarchical Multiple Regression of the School Performance Score (SPS) on Independent Control Variables for Model 2.
Original Construct Configuration of the SISQ .2 Latent Construct Configuration of the SISQ .3 Improvement Recommendations to the SISQ .132 x LIST OF FIGURES 1. Key Components of the School Accountability System in Louisiana. SPS Indicators with Corresponding Weighting Factors. Schein’s Levels of Culture .53 xi LIST OF ACRONYMS APA: American Psychological Association AYP: Annual Yearly Progress CFA: Confirmatory Factor Analysis CRT: Criterion Referenced Test DAT: District Assistance Team EFA: Exploratory Factor Analysis GLE: Grade Level Expectation LDE: Louisiana Department of Education LEAP21: Louisiana Education Assessment Program for the Twenty-First Century MSA: Measure of Sampling Adequacy NAEP: National Assessment of Education Progress NCATE: National Council for Accreditation of Teacher Education NCES: National Center for Education Statistics NCLB: No Child Left Behind Act of 2001 NRT: Norm Referenced Test PCA: Principal Components Analysis SAM: School Analysis Model SBESE: State Board of Elementary and Secondary Education SIP: School Improvement Plan SIS: School Information System SISQ: School Analysis Model Instructional Staff Questionnaire SPS: School Performance Score TIMMS: Third International Mathematics and Science Study USDOE: United States Department of Education xii ABSTRACT This study assessed the construct validity of the School Analysis Model (SAM) Instructional Staff Questionnaire.
Construct validation was necessary for several reasons. First, it has not been possible to obtain evidence of the latent factor structure of this key component of the School Analysis Model (SAM). A factor analysis using data collected with the questionnaire was conducted to assess and identify the underlying factor structure of the instrument. Second, there is no evidence that the constructs measured by the SAM are associated with attributes of school performance further empirical analysis was done to determine if latent constructs contained within the SAM Instructional Staff Questionnaire accounted for a significant proportion of variance in school effectiveness beyond that accounted for by the control variables.
The eight-factor solution of the SISQ was found to be the best representation of the data based on factor loadings, scale alpha reliability estimates, conceptual cohesiveness, and number of items retained. Correlation analyses were conducted to assess the relationship between the SISQ latent factors and the control variables. Findings indicated a significant inverse relationship was found to exist between a school’s SPS and poverty. Additionally, an inverse relationship was found to exist between a school’s SPS and the size of a school.
Several of the latent factors exhibited a relationship to the control variables as well as to other latent factors. Hierarchical multiple regression analysis was conducted in order to determine whether a combination of the latent SISQ factors account for a significant proportion of variance in school effectiveness, as measured by the school SPS. Model 1 indicated that xiii the control variables explained approximately 56% of the variance in SPS. Model 2 indicated that the SISQ latent factors increased the proportion of variance explained by 11%.
The results of this study indicated that the SISQ scales did not account for a significant proportion of the variance in SPS scores and therefore, there is substantial room for improvement in the SISQ as a measurement instrument. Results suggest that construct validation should be of primary concern in the development of measures used to evaluate and guide school improvement efforts. xiv CHAPTER ONE INTRODUCTION Reform in education has been a major and controversial political topic nationwide for the past two decades. Louisiana plays a part in this larger movement and is cited as having made significant progress in improving student and school performance (Education Week, 2004).
The progress made thus far is largely due to changes in state education policy that provided for a variety of strategies designed to improve student and school performance in the form of rigorous content standards, new criterion referenced high stakes tests, and school and district accountability measures that include sanctions and rewards as well as increased attention to teacher quality. Currently there are 800 public elementary schools, 294 middle schools, 251 high schools, and 131 combined or K-12 schools in Louisiana, including 13 charter schools and 395 parochial/private schools (Louisiana Department of Education, 2003). A continued focus on accountability measures with a renewed interest in school climate factors, in addition to teacher quality, promise a robust continuation of the reform agenda. Future efforts to improve Louisiana’s education system and provide a quality education to all of its citizens, depends on the continued commitment and resolve of stakeholders.
To understand where the state reform agenda is headed, an understanding of what brought the state to this point is important. Reform is not a hermetic process, it never happens in a vacuum. Any study of education in Louisiana requires a fundamental understanding of the unique and illuminating history of education in the state. The following section provides a brief synopsis of the origins of education in Louisiana; it has been a long hard journey.
1 Historical Context of Education in Louisiana Louisiana, which consists of 64 parishes and 86 public school systems, is a state with an exceedingly diverse population. Known for its rich cultural heritage and notoriously flamboyant political history, it is a state with an interesting history in education as well. Unfortunately, that history has not always been productive or without harsh criticism. Currently, Louisiana boasts one of the best accountability systems in the country (Education Week, 2004).
Reform of education in Louisiana has local and state support and is a political priority.