MINISTRY OF EDUCATION AND TRAINING HUE UNIVERSITY COLLEGE OF FOREIGN LANGUAGES ----------------- HOANG THI HUONG AN INVESTIGATION INTO TEACHERS' PERCEPTIONS OF LANGUAGE TEST VALIDITY AND THEIR PRACTICE IN DESIGNING TESTS AT SOME UPPER-SECONDARY SCHOOLS IN QUANG TRI MA THESIS IN THEORY AND METHODOLOGY OF ENGLISH LANGUAGE TEACHING HUE, 2014 MINISTRY OF EDUCATION AND TRAINING HUE UNIVERSITY COLLEGE OF FOREIGN LANGUAGES --------------------------------- HOANG THI HUONG AN INVESTIGATION INTO TEACHERS' PERCEPTIONS OF LANGUAGE TEST VALIDITY AND THEIR PRACTICE IN DESIGNING TESTS AT SOME UPPER-SECONDARY SCHOOLS IN QUANG TRI MA THESIS IN THEORY AND METHODOLOGY OF ENGLISH LANGUAGE TEACHING CODE: 60.11 SUPERVISOR: TRUONG BACH LE, D. HUE, 2014 i BỘ GIÁO DỤC VÀ ĐÀO TẠO ĐẠI HỌC HUẾ TRƯỜNG ĐẠI HỌC NGOẠI NGỮ ------***----- HOÀNG THỊ HƯƠNG TÌM HIỂU NHẬN THỨC CỦA GIÁO VIÊN VỀ TÍNH GIÁ TRỊ CỦA BÀI KIỂM TRA NGÔN NGỮ VÀ THỰC TẾ THIẾT KẾ ĐỀ KIỂM TRA CỦA GIÁO VIÊN Ở MỘT SỐ TRƯỜNG THPT TRÊN ĐỊA BÀN TỈNH QUẢNG TRỊ LUẬN VĂN THẠC SĨ LÝ LUẬN VÀ PHƯƠNG PHÁP DẠY HỌC BỘ MÔN TIẾNG ANH MÃ SỐ: 60.11 NGƯỜI HƯỚNG DẪN KHOA HỌC : TS. TRƯƠNG BẠCH LÊ HUẾ, 2014 ii STATEMENT OF AUTHORSHIP I hereby acknowledge that this study is mine. The data and findings discussed in the thesis are true and have not been published elsewhere.
Author HOANG THI HUONG iii ABSTRACT This study attempts to investigate teachers' perceptions of language test validity and their practice in designing tests at some upper-secondary schools in Quang Tri province. A combination of quantitative and qualitative methods, including questionnaire, interviews and test sample evaluation, was adopted to get data for this paper. The data analyzed and discussed provide an overview on teachers' perceptions of language testing in general and the validity of language tests as well as basic principles of test design practice. In addition, the evaluation of the test samples using criteria in accordance with language test validity helps show how the teachers design English tests in reality.
The findings indicate that while the teachers have good perceptions of language tests validity and acknowledge the importance of test design practice, their practice of test design is not as good as their perceptions imply. The mismatch between the teachers' conceptions and their practice reveals a need for formal training for teachers in the field of language testing and test design practice. iv ACKNOWLEDGEMENTS Above all, I would like to thank my supervisor, Dr. Truong Bach Le, for his valuable instruction, advice and encouragement throughout this study.
Without his practical guidance, this research paper would not have been completed. I would also like to express my gratitude to my teachers at Hue University of Foreign Languages, who listened and supported my efforts throughout the process. It would be a big mistake for me not to thank my colleagues and friends who heartedly participated in completing the questionnaire, answering the interviews and sending me the test samples. Their substantial assistance greatly contributed to the accomplishment of this research.
Last but not least, I would like to thank my family for their understanding and endless support from the very beginning of my studies to the end of this project. I am truly blessed to have such considerable encouragement and assistance. v TABLE OF CONTENTS Pages SUB COVER PAGE. i, ii STATEMENT OF AUTHORSHIP.
v TABLE OF CONTENTS. vi LIST OF ABBREVIATIONS. ix LIST OF TABLES AND FIGURES. Aims of the Study .3 Significance of the study.
Organization of the thesis. 4 CHAPTER 2: LITERATURE REVIEW. An overview of language testing. The development of language testing.
Purposes of language testing. Common types of English language tests. The designing of language tests. Content validity and Face validity.
Criterion-related validity. Concurrent validity and Predictive validity. The relationship between teachers' perceptions and practice. The situation of language testing at upper-secondary schools in Quang Tri province.
The use of language tests. The designing of language tests. 25 CHAPTER 3: RESEARCH METHODOLOGY .4, Test evaluation sheet, curriculum, test matrices. Procedure of data collection and analysis.
33 CHAPTER 4: FINDINGS AND DISCUSSION. Teachers' perceptions of language test validity. Teachers' beliefs regarding language testing and the use of language tests. Teachers' conceptions of language test validity.
Teachers' test design practice in accordance with language test validity 43 4. Teachers' beliefs regarding language test design. How teachers design language tests in reality. Results from the questionnaire and the interviews.
Results from test sample evaluation. The relationship between teachers' perceptions of language test validity and their practice of test design. 62 CHAPTER 5: CONLUSION AND IMPLICATIONS. Restatement of the Research Aims.
Summary of the Findings. What are teachers' perceptions of the validity of language tests?. How do they conduct the designing of language tests in reality?. What is the relationship between their perceptions and their practice?65 5.
Significance of the Study. Implications of the Study. Limitations of the Study. 72 viii LIST OF ABBREVIATIONS SP: Specification SPSS: Statistics Package for the Social Sciences TLU: Target Language Use MOET: Ministry Of Education and Training ix LIST OF TABLES AND FIGURES Table 3.1: Profile of teacher participants Table 3.2: Distribution of the questionnaire items Table 3.3: A testing program checklist 1 Table 3.4: A testing program checklist 2 Table 3.5: A testing program checklist 3 Table 3.6: A testing program checklist 4 Table 4.1: Teachers' ideas on Drawing out test specifications Table 4.2: Teachers' ideas on Defining the construct Table 4.3: Extracts from the questionnaire and the test evaluation sheet Figure 4.1: The importance of language testing at schools Figure 4.2: Teachers' purposes of using language tests at schools Figure 4.3: Common reasons for language tests' popularity at schools Figure 4.4: Teachers' understanding of the definition of language test validity Figure 4.5: Teachers' understanding of different types of language test validity Figure 4.6: Teachers' belief regarding the importance of language test validity Figure 4.7: Reasons for Validity's importance to language tests Figure 4.8: Teachers' conceptions of the importance of test design Figure 4.9: Teachers' practice of test design x CHAPTER 1 INTRODUCTION This chapter introduces the research study.
It deals with (1) theoretical background, (2) the aims of study, (3) the research questions, and (4) the preferred organization of the thesis. Theoretical background Over the years, language testing and the use of language tests have become essential parts in the process of teaching of most language teachers (Bachman & Palmer, 1996). The measurement of students' language understanding and use through tests is considered necessary in assessing their language abilities (Hughes, 2003). Tests used for evaluating students' language competences vary in purposes and forms.
They can be used to identify different skills of students in understanding and applying the knowledge achieved. They can be either in the forms of oral tests, written tests, computer-based tests or Internet-based tests. Despite whatever purposes or forms, language tests need to meet certain requirements so that they can help teachers make suitable inference of the test-takers' ability (Bachman & Palmer, 1996). In other words, language tests must bear certain qualities to be useful for purposes of language teaching and learning.
In Vietnam, the use of tests to evaluate students' abilities has been a tradition and a norm for the whole education system and for almost every subject in schools from the lower to higher grades. Considered as a useful tool, tests are supposed to reveal students' educational achievements and evaluate the quality of school systems (Fulcher & Davison, 2007). This is especially true in the context of Vietnam, where the class size is big (up to 50 students), it is really difficult for teachers to make any inference of their students' progress without the help of formal tests. Therefore, teaching and testing English are in the same situation.
It is a fact that teaching and learning English in the Vietnamese context is examination- oriented, and although “By the mid-1980s, the language testing field had begun to focus on designing communicative language-testing tasks” (Brown, 2003, p. 10), it 1 is assumed that the tests used in Vietnam are mainly to assess students' knowledge of grammar patterns and lexical items. In reference to Vietnam’s National Foreign Language 2020 Project (MOET, 2008), whose mission is to implement the Government’s Decision 1400 (2008) on building national foreign language capacity, language testing and assessent are also renovated. With the aim to improve and ensure the quality of foreign language teaching and learning on a national scale, the methods of assessments, in which testing is an important part, must be modified.
The designing of language tests used to be misunderstood that it must follow the "best" models which had been set up by test-designing experts (Bachman & Palmer, 1996). This problematic assumption refers to the best tests for all purposes and situations. However, it has been proved to be inappropriate by previous studies. (2004) also admits some obvious shortcomings of what are called standardized tests.
It is clear that designing tests must be based on relevant goals and contexts with an understanding of its intended test-takers' characteristics. Moreover, a good perception on the usefulness of language tests is vital as it can serve as a base for test designers to achieve their purposes in testing, and then lead to positive washback on the process of teaching and learning. So, what is the usefulness of language tests? How do teachers identify the qualities of a good test? Do teachers really find it easy to design tests in accordance with their requirements? These questions have globally attracted great concern. Recently, there have been a number of studies researching issues relating to language testing and test designing.
Some of them investigated the practice of designing and using tests, such as Trinh Luong Thu (2009), Le Thi Thanh Hai (2010), Vo Thi Dieu Hien (2012) and Nguyen Duc Ai Nhat (2012). These theses have focused on teacher's perceptions of language tests qualities in general and their practice in designing tests. The matter is that they failed to make a comparison between teachers' perceptions and their practice through thoroughly examining the tests designed and used. Aims of study As a language teacher for nearly seven years, the author has designed quite a great number of tests for my students.
Unfortunately, she, as well as her colleagues, has not had any formal training in designing and developing language tests. From the fact that there exist misconceptions of language testing which negatively influences the designing and development of tests, and from what she has experienced during her own teaching and testing practices, the researcher wishes to conduct this project. The project aims to investigate teachers' perceptions of the usefulness of language tests in terms of validity, and how they design tests for students in reality. Significance of study From her investigation, it is hoped that we can build up an overview of teachers' insight and their practice of language testing.
In other words, her findings will provide a picture of the extent to which teachers understand and implement language testing in accordance with its validity. The examination of those issues can also serve the purpose of helping language teachers get a deeper understanding of required qualities of language tests and then seriously apply what they have learnt to the designing of tests for their students. Research questions The paper aims to investigate the perceptions of upper-secondary school teachers in the validity of language tests and their practice in designing tests for students. The author assumes that a certain number of teachers do not have an adequate awareness of the usefulness of language tests from the perspective of test validity.
This inadequate perception may lead teachers to some limitations in designing tests, and even those who tend to be competent in language test designing may provide some invalid and unreliable tests due to some other factors.