Author: Veeravagu, Jeyamahla; Muthusamy, Chittra; Marimuthu, Rasaya; Michael, Angelina Subrayan
Date published: May 1, 2010
Journal code: CDSS
résolues. Cet article étudie la relation entre le niveau des processus de pensée dans les questions de compréhension et de la performance des étudiants. Les résultats indiquent que le niveau de questions conçues selon la taxonomie de Bloom influence sur la performance des élèves dans leurs réponses aux questions de compréhension. Les résultats concluent qu'il y a une relation entre le niveau de processus de pensée nécessaire et la capacité des élèves à répondre à ces questions correctement. Ce document fournit une base commune pour des discussions plus approfondies sur la compétence de l'anglais des élèves de premier cycle, ainsi que les recommandations sur les techniques qui pourraient être utilisées pour traiter des questions d'un plus haut niveau.
Mots-clés: niveau de processus de pensées; questions sur la compréhension écrite; performance; taxonomie de Bloom; questions de choix multiple (MCQs)
Reading is an essential skill critical to most, if not all, academic learning and success at tertiary level. Reading comprehension then, is a thinking process by which a reader selects facts, information or ideas from printed materials; determines the meanings the author intended to transmit; decides how they relate to previous knowledge and judges their appropriateness and worth for meeting the learner's own needs and objectives.
To encourage meaningful understanding, apart from the above, a learner needs to understand and remember texts by inferring, elaborating ideas, and discarding unimportant details (Garner, 1988). Such tasks engage cognitive processes that require learners to follow and respond to a message from a writer who is distant in space and time (Davis, 1995). Logically, active and thoughtful reading procedures should lead learners to critically analyze and think of the text, resulting in the reconstruction of knowledge. Many researchers advocate this concept of reading as a source for critical thinking engagement with texts because of its potential to facilitate, re-enact and reconstruct knowledge that produces meaning and understanding i.e. comprehension (Fielding & Pearson, 1994).
Comprehension includes all the skills and abilities necessary for literal, inferential and critical reading. Thus, reading comprehension questions are used extensively in testing language proficiency courses in Univesiti Teknologi MARA (UiTM). This study intends to analyze UiTM undergraduates' ability in answering reading comprehension questions at different levels of thinking and the test scores obtained by them.
The purpose of this study is to find answers to the following research questions;
1 . How did the students perform in a reading comprehension paper?
2. How did the students score in answering different levels of thinking processes (Bloom's Taxonomy) in this reading comprehension paper?
2. LITERATURE REVIEW
Instructional assessment plays an essential role in the progress of the instructional process. It is an essential part of the teaching-learning process. Students' learning can be measured by different procedures. One of these procedures, which are widely used by teachers in the classroom, is the achievement test. Good achievement tests are supposed to measure different levels of learning. Bloom (1956), suggested six different cognitive stages in learning from the simple recall or recognition of facts, as the lowest level, to the highest order, which is evaluation.
2.2 Objectives Of Reading Tests
Many reading tests still operate on the principle that meaning is text -immanent, and that the leaner's job is to find the same meanings that the test designer found. The problem with testing comprehension is that they test only whether students are able to think what we want or expect them to think. Yet, they do not lead to an accurate assessment of what students really are thinking as they read. In fact, 'forced choice' item type such as multiple choice usually produce the most problems among the best readers, who invariably find some logical flaw or are confused by the only partial adequacy of the possible answers provided (Horowitz, 1991).
Despite what experts have said about the nature of reading, many teachers measure comprehension by how well students recall the details of what they have read (Allington, 2001). Thus, most students are judged as proficient readers because they can answer questions related to the factual information in the text. However, when the assessment focuses on critical reading and responding to text, only a few students demonstrate even minimal proficiency.
2.3 Multiple choice testing
Ultimately, there is no one ideal assessment process. Multiple choice question (MCQ) testing has its advantages and disadvantages. Chan and Kennedy (2002) suggest MCQs responses are easily scored with accuracy and objectivity. Importantly, the tests do not openly disadvantage students with weak reading skills to the same degree as essay questions. Misreading one MCQ may lose the student a small percentage; however, misreading an essay question can result in substantial losses, Overall, MCQs can provide increased breadth and depth of coverage of material as opposed to essays that may concentrate on a focused area.
Multiple choice tests appear to be controversial and the major problems stem from poorly or in appropriately constructed test items (Paxton, 2000). It is recognized that creativity cannot be tested and this disadvantage reflects Bloom's taxonomy of combining ideas to form a new whole (Bloom, Engelhart, Fürst, Hill & Krathwohl,1956). However, the other five components of Blooms taxonomy: knowledge, comprehension, application, analysis and evaluation can be tested through MCQs.
MCQs are able to provide information about student's higher levels of understanding rather than the notion that MCQ tests higher order learning ideals (Higgins & Tatham, 2003). If measures are correctly designed, the tests are able to assess the depth and breadth of students' knowledge (Epstein, Lazarus, Calvano & Matthews, 2002). However, some students will engage in rote learning irrespective of the assessment method.
2.4 MCQ testing
Possession of an adequate knowledge base was once considered as unimportant, as knowledge is changing so rapidly. The ability to find out and solve problem was stressed as being more important. According to Norman (1996), problem-solving and competence are not generic and are dependent on individual cases, tasks, situations, problems and, crucially, is knowledge-dependent .
A longstanding criticism of the validity of MCQs is that testing cognitive knowledge does not guarantee competence as professional competence integrates knowledge, skills, attitudes and communication skills. However, decades of research have shown that knowledge of a domain is the single best determinant of expertise (Glaser, 1984). MCQs are, therefore, a legitimate method of competence testing, as cognitive knowledge is best assessed using written test forms (Downing, 2002).
While MCQs are expressly designed to assess knowledge, well-constructed MCQs can also access taxonomically higher-order cognitive processing such as interpretation, synthesis and application of knowledge rather than testing recall of isolated facts (Case & Swanson, 2001). However, 'higher-order' MCQs still require cognitive knowledge and may not be any more valid but their practicality makes them more acceptable to examinees and examiners .However, a fair MCQ-based test is much more than a statistically reliable test of cognitive knowledge.
The Research Design used for this study is the Content Analysis Research and correlation study of the results between the scores of each item against the level of thought processes of the Bloom Taxonomy. This is a quantitative study; this type of study was chosen because the quantitative data appears to be easier to interpret because it is more specific and explicit rather than implicit in nature (Sulaiman, 2004).
3.1 The Subjects
A total of fifty Part 3 diploma students were chosen for this research. Convenience sampling was used in this study because the participants were willing and available to be studied. These part 3 students also have similar academic background. Their English language proficiency level ranges from low intermediate to high intermediate. These students have been exposed to answering reading comprehension passages in Part 1 and Part 2.
3.2 The Design
Correlational Research design was used for this study. This correlational explores the relationship between the students' performance and the level of thinking process of the Bloom Taxonomy in answering a reading comprehension paper. This is a quantitative study; this type of study was chosen because the quantitative data appears to be easier to interpret because it is more specific and explicit rather than implicit in nature (Sulaiman, 2004). Test scores would be collected and analyzed. This paper attempts to measure the students' reading comprehension scores against the level of thinking processes of the RCQs on questions formation advocated by the Bloom Taxonomy.
3.3 The Instrument
A set of reading comprehension question was used. Thirty-five multiple choice items were tested in this paper. 7 questions on information transfer and 28 questions designed from 4 different reading passages. The test scores were collected and analyzed. The students' reading comprehension scores against the level of thinking processes of the questions using Bloom Taxonomy was analyzed quantitatively.
Table 1 shows the total percentage score of the students with the correct answers for each question in the reading comprehension paper used in this study. The students' performance in this paper varies according to the levels of thinking process. Students performed better in questions with low level thinking process (knowledge, comprehension and application) compared to high order questions.
From Table 2, only 66.0 % of students responded the Knowledge level questions correctly. As for the Comprehension level, 57.2% of students got them right. 47.0 % of the students were able to answer the Application level questions correctly. Only 48.4% of students managed to answer the Analysis level questions correctly. As for the Synthesis questions, 33.6% of the students got them right. There was only one question tested on the evaluation level with only 28.0 % of the students were able to answer accurately.
The findings indicate that the level of thinking processes advocated by Bloom taxonomy has influenced the performance of the students in the reading comprehension paper. It seems that the students face difficulties when answering higher level questions especially questions at level 4, 5 and 6 (analysis, synthesis and evaluation). It also influences the students' ability to answer higher order questions as compared to lower order questions. The findings conclude that there's a relationship between the level of thinking and the students' ability to answer them correctly.
Researchers have found that teaching reading strategies is important to developing increased student comprehension. At the same time, they have found many teachers lack a solid foundation for teaching these reading comprehension strategies (National Reading Panel, 2005).Teachers must employ comprehension strategies to teach reading comprehension. Comprehension monitoring helps students what they understand or do not understand while reading a text. It also helps them to use "fix-up" strategies such as re-reading for a particular purpose or adjusting reading speed as related to text difficulty. Answering a variety of questions from literal to application types during pre-reading, reading and post reading provides students with a purpose and focus for reading. Asking these questions during the process improves student's active engagement with text.
5.1 Pre-Reading, While Reading and Post Reading
According to Chia (2001), some students report that they have no problem with understanding both words and sentence structures of the paragraph, but they cannot reach satisfactory interpretation of the text. Hence, pre- reading, activities should emphasize methods ofmerging the students, text and content. To assist the reading process, recall related prior knowledge, preview and predict what the text will be about. We need to build their background by activating appropriate prior knowledge through questioning about the text. Wallace (1992) proposes one very popular kind of pre-reading task is "brain storming". Brainstorming has many advantages. Firstly, it requires little preparation; second, it allows student considerable freedom to bring their own prior knowledge and opinions to bear on a particular issue; and third, it can involve the whole class.
Previewing motivates students to read. Swaffar et. al. (1991) pointed out the benefits of previewing techniques that allow students to formulate hypotheses about the text. Teachers ask students to recall and consider prior personal experiences that are relevant to the text and help to build the necessary background knowledge about the text.
While-reading activities should enable students to monitor their comprehension through a variety of strategies and experiences. Teachers should monitor comprehension by teaching students to guess meaning of new words using the cueing system. Post-reading activities helps students to review their understanding of text, relate new ideas to their schemata, revise their thinking, apply the information to other texts and remember crucial learning for future application.
5.2 Vocabulary Building
Vocabulary is essential for getting meaning from text. Thus, it is recommended that teacher should consider using many activities before reading strategies to improve students' vocabulary. As a student begins to read, reading vocabulary stumbled upon in texts is mapped onto the oral vocabulary he/she brings to the task. When the word is not in the students' oral vocabulary, it will not be understood when it occurs in print. Vocabulary occupies an important ground in learning to read. Reading vocabulary is important in the comprehension processes of students.
Christen and Murphy (1991) insist that research clearly emphasizes that for learning to occur, new information must be incorporated with what the students already knows. They feel that teaching vocabulary as a pre-reading step is an instructional intervention that should be considered when students lack the prior or background knowledge to read in a content area.
5.3 Breaking up of text
We could also utilize the breaking up the text strategy. A long text may appear daunting to students whose reading is not very skilled or speedy. When you ask the class to read it silently, the slower students will feel even more inadequate, while the quicker ones will finish long before the time you have allowed. This is a nuisance and may be a problem if discipline is shaky, since it gives the better students time to create disturbances. Kawabata (2007) agrees this method allows the students to analyse more detail in each section, obtain specific information more closely, combine information of the sections, and understand the main idea of the text. Skimming, scanning, and breaking up the text are introduced as group activities to develop and confirm the students' understanding further.
Some people read better than others but teachers can reduce the effects of the differences by dealing with the text in several short sections instead of all at once. In this way, the better students may be kept waiting only a couple of minutes for each section, instead of a much longer time if the slower ones have to finish the whole text.
Not being able to perform efficiently in English affects a substantial number of students in higher institutions, especially when most subjects are taught in English using textbooks written in English. Weak students are also generally not prepared to "perform" in the language because in-depth thinking processes are required in the study of the sciences and arts at tertiary level. They find it difficult to understand the difficult texts in books and this will affect their performance in their core subjects. At this level, they need to speak, write and critically analyze in English to be well informed and competent, failing which they will resort to memorizing and copying without fully understanding the contents. Higher-order thinking occurs when students look beyond the surface of the text they are reading to figure out an answer or to attain comprehension. Making predictions, drawing conclusions and making inferences are examples of reading strategies that typically elicit higher-order thinking. While there is a recognized demand to have higher-order thinking practice in the classroom, there is also a recognized instructional struggle with bringing higher-order thinking to life in the classroom.
Allington, R. (2001). What really matters for struggling readers: Designing research-based programs. New York: Longman.
Bloom, B. (ed.) (1956). Taxonomies of education objectives: The Classification of Educational Goals; Handbook I, Cognitive Domain. New York: Mckay.
Bloom, B.S., Engelhart, M.D., Fürst, E.J., Hill, W.H. & Krathwohl, D.R. (1956). Taxonomy of Educational Objectives. Longmans Green: New York.
Case, S.M. & Swanson, D.B. (2001). Constructing Written Test Questions for the Basic and Clinical Sciences, 3rd edn. Philadelphia: National Board of Medical Examiners.
Chan N. & Kennedy, P.E. (2002). Are multiple-choice exams easier for economics students? Southern Economic Journal, Vol 68, Issue, 4.
Chia, H.L. (2001). Reading Activities for Effective Top-down Processing. FORUM Vol 39, NoI January March p. 22.
Christen, W L. & Murphy T. J. (1991). Increasing Comprehension by Activating Prior Knowledge. ERIC Digest. Bloomington, In: ERIC Clearinghouse on Reading, English, and Communication.
Davis, F. (1995). Introducing Reading. London: Penguin Books.
Downing, S.M. (2002). Assessment of knowledge with written test formats, in: G. Norman, C. Van Der Vleuten & D. Newble (Eds) International Handbook of Research in Medical Education, Vol. 2.
Epstein, M.L., Lazarus, A.D., Calvano, T.B. & Matthews, K.A. (2002). Immediate feedback assessment technique promotes learning and corrects inaccurate first responses. The Psychological Record, Vol., 52, Iissue 2, pp. 187-2002.
Fielding, L. & Pearson, P. (1994). Reading Comprehension: What Works. Educational Leadership, 51(5), 154-162.
Garner, R. (1988). Metacognition And Reading Comprehension. New Jersey: Prentice-Hall.
Glaser, R. (1984). Education and thinking: the role of knowledge, American Psychology, 39, pp. 193-202.
Higgins, E & Tatham, L., (2003). Assessing by multiple choice questions tests. UKCLE Resource Bank. Manchester Metropolitan University, Manchester.
Horowitz, D. (1991). Evaluating learners' performance. In Kern, R. Literacy and language Teaching. Oxford University Press.
Kawabata, T.(2007). Teaching Second Language Reading Strategies. The Internet TESL Journal, Vol. XJII Retrieved March 29, 2009 from hnp://iteslj.org/Techniques/Kawabata-ReadingStrategies.html.
National Reading Panel. (2005). Comprehension III teacher preparation and comprehension strategies instruction. (Chap. 4). Retrieved March 10, 2009 from http://www.nichd.nih.gov/publications/nrplch4-111.pdf
Norman, GR. (1996). Multiple choice questions, in: S. Shannon & G.Norman (Eds). Evaluation Methods: A Resource Handbook, 3rd edn. (Hamilton, Canada), pp. 47-54.
Paxton, M., (2000). A linguistic perspective on multiple choice questioning. Assessment and Evaluation in Higher Education, Vol 25, Issue 2, pp. 109-130.
Sulaiman Shamsuri (2004). Research Methods for the Social Sciences Made Simple. Klang: DSS Publishing Enterprise.
Swaffar, J. , Arens,K, & Byrones,H. (1991). Reading for Meaning: An Integrated approach to language learning. Englewood Cliffs, NJ: Prentice Hall.
Wallace, C. (1992). Reading. Oxford : Oxford University Press.
1 Academy of Language Studies, Universiti Teknologi MARA (UiTM) Dungun 23000, Terengganu, Malaysia. Email: email@example.com.
2 Academy of Language Studies, Universiti Teknologi MARA (UiTM) Dungun 23000, Terengganu, Malaysia. Email: firstname.lastname@example.org.
3 Academy of Language Studies, Universiti Teknologi MARA (UiTM) Pulau Pinangl3500, Malaysia. Email: email@example.com.
4 Academy of Language Studies, Universiti Teknologi MARA, Malaysia, Shah Alam 40450, Malaysia. Email: firstname.lastname@example.org.
* Received on March 14, 2010; accepted on June 2, 2010