VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST – GRADUATE STUDIES SUBMITTED BY: TRẦN THỊ THU HƯƠNG A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts EVALUATING THE VALIDITY OF THE FINAL ACHIEVEMENT TEST FOR SECOND – YEAR NON – MAJOR STUDENTS AT ELECTRONIC – ELECTRICAL ENGINEERING DEPARTMENT, NAM DINH UNIVERSITY OF TECHNOLOGY EDUCATION (Đánh giá độ giá trị của bài kiểm tra cuối kỳ cho sinh viên không chuyên tiếng Anh năm thứ hai tại khoa Điện – Điện tử, Trường Đại học Sư phạm Kỹ thuật Nam Định) M. MINOR THESIS Field: Language Teaching Methodology Code: 60 14 10 HANOI, 2011 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com VIETNAM NATIONAL UNIVERSITY, HANOI UNIVERSITY OF LANGUAGES AND INTERNATIONAL STUDIES FACULTY OF POST – GRADUATE STUDIES SUBMITTED BY: TRẦN THỊ THU HƯƠNG A thesis submitted in partial fulfillment of the requirements for the degree of Master of Arts EVALUATING THE VALIDITY OF THE FINAL ACHIEVEMENT TEST FOR SECOND – YEAR NON – MAJOR STUDENTS AT ELECTRONIC – ELECTRICAL ENGINEERING DEPARTMENT, NAM DINH UNIVERSITY OF TECHNOLOGY EDUCATION (Đánh giá độ giá trị của bài kiểm tra cuối kỳ cho sinh viên không chuyên tiếng Anh năm thứ hai tại khoa Điện – Điện tử, Trường Đại học Sư phạm Kỹ thuật Nam Định) M. MINOR THESIS Field: Language Teaching Methodology Code: 60 14 10 Supervisor: Phạm Lan Anh, M.A HANOI, 2011 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com vi TABLE OF CONTENTS DECLARATION. iii LIST OF FIGURES, TABLES AND CHARTS.
iv LIST OF ABBREVIATIONS. v TABLE OF CONTENTS. vi CHAPTER 1: INTRODUCTION. Scope of the study.
Aims of the study. Methods of the study. Design of the study. 3 CHAPTER 2: LITERATURE REVIEW.
Relationship between teaching, learning and assessment. Purposes of formative and summative assessments. Achievement tests and their characteristics. Characteristics of a good EGP test.
Characteristics of a good ESP test. Relationship between reliability and validity. Reasons for choosing face validity. Some measures to increase face validity.
18 LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com vii CHAPTER 3: THE STUDY. English learning and teaching at Nam Dinh University of Technology Education. The English teaching staff. Objectives of the English course.
Checklist of the course book. Objectives of the final test. Difficulty level and discrimination of the final test. English testing at Nam Dinh University of Technology Education.
The current final achievement test. Interview and informal discussion. Data analysis of survey questionnaires and interviews. Data analysis of the administration of the test.
Data analysis of the format of the test. Data analysis of the logistics of the test. Data analysis of face validity of the test. Data analysis of general opinion about the test.
Data analysis of reading comprehension task. Data analysis of grammar knowledge task. Data analysis of translation task. Discussion and findings.
Similarities in teachers and students’ perception. 36 LUAN VAN CHAT LUONG download : add luanvanchat@agmail. General opinion about the test. Reading comprehension task.
Differences in teachers and students’ perception. Suggestions to improve the final achievement test. XIII LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com iv LIST OF FIGURES, TABLES AND CHARTS Figures Figure 1: Three Considerations for Test Choice. Figure 2: The Scope of Impact of Language Tests.
Figure 3: Relationship between reliability and validity. Tables Table 1: ESP syllabus content allocation. Table 2: Specification of test 12. Table 3: Teachers and students’ comments on the format of the test.
Table 4: Teachers and students’ comments on the administration of the test. Table 5: Teachers and students’ comment on the whole test. Table 6: Teachers and students’ comment on students’ reading comprehension ability, theme and instruction of the reading comprehension task. Table 7: Teachers and students’ comment on the grammar task.
Table 8: Teachers and students’ comment on the translation task. Charts Chart 1: Percentage of teachers and students’ comment on what language ability the test mainly intend to measure. Chart 2: Percentage of students and teachers’ opinions on what test items can measure their true ability. Chart 3: Percentage of the results which students get in the test.
Chart 4: Percentage of what test tasks students cannot do. Chart 5: Percentage of teachers and students’ comment on the length of the reading text. Chart 6: Students’ comment on whether or not the reading text is difficult. Chart 7: Teachers’ comment on which types of reading skills are expressed in the reading comprehension task.
Chart 8: Teachers and students’ comment on which students’ ability this translation task requires. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com v LIST OF ABBREVIATIONS NUTE: Nam Dinh University of Technology Education EGP: English for General Purposes ESP: English for Specific Purposes LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com 1 CHAPTER 1: INTRODUCTION 1. Rationale Learning English in Vietnam today is quite popular and its popularity is increasing day by day. This is due to the fact that Vietnam has recently adopted an open-door policy which encourages broadening and improving its relationship and cooperation with other countries in many aspects of life such as diplomatic, economic, cultural, scientific and technological areas.
English is not only a means but also a key to gaining access to latest scientific and technological achievements for a developing country like Vietnam, where modern science and technology is badly needed. Recognizing the necessity of this global language, most of the schools, colleges and universities in Vietnam consider English as the main, compulsory subjects that students must learn. In the English language learning and teaching process, evaluating a test is significant. Test is a part of teaching and learning process because it also provides feedback about the achievement of teaching and learning objectives for those who are involved in the education system.
Moreover, the use and necessity of knowing a person’s language ability is through a language test. In education, especially at the Faculty of Foreign Languages at Nam Dinh University of Technology Education (NUTE), testing the students’ achievement toward teaching objectives is needed. Without an achievement test, it is difficult to see how rational educational decisions can be made. NUTE is a technological university and students’ learning English ability is really low.
Evaluating a test is also one of the methods to improve students’ learning process and results. Although there were many theses mentioned this problem but at NUTE it is still new and very necessary because how to evaluate the test after each semester still receives little attention and up to now the process of test analysis after each examination hasn’t been fully investigated. Consequently, students’ results are getting worse and worse. As a teacher myself, I see that we, teachers at NUTE, just stop at experienced level of test making procedure, test administration, test marking procedure during and after examination.
We do not care testing evaluation from other teachers and students, so the results of test are not still improved. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com 2 Moreover, test’s validity can be seen as an attempt for improving the test quality. Being a measure of students’ achievement toward learning objectives, final examinations must be valid. Validity is one of the characteristics of a qualified test.
Therefore, “Evaluating the validity of the final achievement test for second – year non – major students at Electronic – Electrical Engineering Department, Nam Dinh University of Technology Education” is chosen with the hope that the study will be helpful with the author, the teachers, the test-takers and everyone who is concerned with language testing in general and validity of an achievement test in particular. Due to limit time in collecting students’ scores, this study is different from the previous study. The author only focuses on the face validity of this test. The author hopes that the result of the study can then be applied to improve the current test and to create a new really reliable item bank.
It is also intended to encourage both teachers and learners in their teaching and learning. Scope of the study The scope of this thesis is limited to a research on teachers’ and test-takers’ evaluation of the existing achievement test in terms of its face validity for the second-year non-English major students at Electronic – Electrical Engineering Department, NUTE due to the limitations in time, ability and availability of data. Moreover, it is impossible for the author to cover all used final achievement tests as well as design a sample achievement test for second-year students. Instead, only a test specification for test 12 in semester 3 is presented.
Aims of the study Following the scope of the research above, the aims of this research are: 1. To indentify the English teachers and students’ evaluation of the final existing achievement test (test 12) at NUTE in terms of face validity. To provide suggestions for test designers. Methods of the study In order to achieve the above aims, the study has been carried out as follows: First, the author goes to library to read theory about assessment and testing, achievement test with characteristics of a good achievement test and test validity with a special focus on face validity and some measures to increase it.
From her critical reading, many reference materials have been gathered, analyzed, and synthesized to draw out a LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com 3 theoretical basis to evaluate the current test being used for the 3rd semester students in terms of its face validity. Then, qualitative methodologies involving data collected through survey questionnaires and interviews were employed from both teachers and students at NUTE. Research questions This study is implemented to find answers to the following research questions: 1. What are the teachers’ and test takers’ (students’) perceptions of the final 3rd semester English achievement test at NUTE in terms of its face validity? 2.
What are suggestions to improve face validity of the final 3rd semester English achievement test at NUTE? 1. Design of the study The thesis is divided into four major chapters: Chapter 1: Introduction presents basic information such as: the rationale, the scope, the aims, the method, the research questions and the design of the study. Chapter 2: Literature review reviews theoretical backgrounds on evaluating a test, which includes relationship between teaching, learning and assessment, purposes of formative and summative assessments, achievement tests, characteristics of good EGP and ESP tests, face validity and some measures to increase face validity. Chapter 3: The study is the main part of the thesis showing the context of the study and the detailed result obtained from collected tests and findings in response to the research questions.
Then, the author gives some solutions to improve the final achievement test. Chapter 4: Conclusion offers conclusions and proposes some suggestions for further research on the topic. LUAN VAN CHAT LUONG download : add luanvanchat@agmail.com 4 CHAPTER 2: LITERATURE REVIEW This chapter provides an overview of the theoretical background of the study. It includes three main sections.1 discusses the relationship between teaching, learning and assessment.2 focuses on the purposes of formative assessment and summative assessment.3 gives a brief description of achievement tests, characteristics of a good EGP test and ESP test.
It is then followed by section 2.4 in which face validity is focused.5 suggests some measures to increase face validity. Relationship between teaching, learning and assessment In the relationship between teaching, learning and assessment; curriculum and content standards also play an important role. Curriculum is best characterized as what should take place in the classroom. It describes the topics, themes, units and questions contained within the content standards.
Content standards are the framework for curriculum. Curriculum can vary from programs to programs, as well as from instructors to instructors.