|
|
|
An Investigation of the Effects of Exam Essay Questions On Student Learning in United States History Survey Classes
Sara Brooks Sundberg
University of Central Missouri
| THE AMERICAN HISTORICAL ASSOCIATION argues, in its "Statement of Excellence in Classroom Teaching of History," that written assignments are important components of the excellent history classroom because they ask students to use critical thinking skills. The analysis and synthesis of historical materials that are part of critical thinking in the history classroom can provide students the opportunity to think historically. In other words writing assignments can stimulate students to discover the complexity within historical materials, not just memorize them or order them chronologically. The AHA is also emphatic that objective tests in the history classroom should never be the only or "final measure of student success."1 Writing in the history classroom is not only an important part of the process of learning, it is an effective assessment tool. |
1
|
|
Historians, of course, are not alone in the advocacy of the importance of writing assignments in the classroom as a teaching methodology or as an assessment tool. Since the 1980s colleges and universities all over the country have developed Writing Across the Curriculum [WAC] programs that emphasize the benefits of writing for student learning and assessment. WAC programs, in general, stress that frequent writing in the classroom encourages students to practice higher-order thinking skills. Writing demands organization, analysis and synthesis.2 |
2
|
|
However, writing assignments are time consuming to critique and grade. For a master's degree granting institution, such as the one considered in this study, the average class size for an introductory level United States history course is 150 students, compared to thirty-three students in an introductory survey in a private institution, typically a liberal arts college.3 In non-doctoral institutions like the one in this study the 4/4 course load nearly matches the 4.2 courses per semester at two year colleges. It is not likely that these numbers will decline anytime in the near future. Undergraduate student enrollments, overall, are rising, as are the number of history majors.4 Given these constraints, it is not surprising that a recent survey indicated that instructors of introductory level history classes in United States history classes at public universities tended to make more use of multiple-choice questions and that they gave less weight to essay writing on finals than their counterparts in private institutions.5 This would seem to fit with research demonstrating that class size and learning are inversely related.6 Teacher and scholar Wilbert J. McKeatchie writes, for example, that "in general, large classes are simply not as effective as small classes for retention of knowledge, critical thinking, and attitude change."7 On the other hand, class size makes less difference if the goals are more narrowly defined in terms of mastering just factual content.8 |
3
|
|
There are, of course, many kinds of writing exercises. Some, like "minute papers" that ask students to write brief responses to questions that require specific answers are one type of writing assignment adapted to large classes. They are useful as a type of formative assessment and, most importantly, in terms of time commitments on the part of the instructor, minute papers can be graded in a minimal amount of time. In some cases minute papers are even peer graded.9 However, minute papers do not fulfill the same learning objectives as longer essays. They usually do not require higher order thinking skills. Long essays are, most often, analytical writing assignments that ask students to assemble, evaluate and/or interpret, and synthesize materials in order to address an assignment. Students are encouraged to engage in a cognitive process that develops their ability to think historically. Peer grading is problematic with this type of assignment because most students are still developing these skills themselves.10 |
4
|
|
A study as long ago as the 1980s suggested that the consequences of not having students do written work were significant for educators in history classrooms. Using two groups of students from a traditional lecture-based history survey course, the investigators paired one group of students with an English composition class that required multiple writing assignments related to content from their history survey source. The other group participated in English composition class but their assignments were not connected with their history survey.11 Students from the group of history students who participated in the English composition classes with writing assignments connected to their history class "fared significantly better" on written assessments of their learning in the history survey course and in their attitudes toward learning in history.12 This study was conducted in an ideal climate for history learning in which students received instruction in writing and history and engaged in multiple writing projects for their class in English that utilized content from their survey course in history. |
5
|
|
This study was more limited. It compared the learning outcomes in sections of a course in which only objective questions were used, with the learning outcomes in sections of the same course that used essay questions on all exams. I decided to explore whether or not the simple addition of essay questions in examinations (whatever the other benefits, such as improved thinking skills) increased the learning of the sort normally tested by objective questions alone. It is important to note that I did not simply add essay exams in some sections, I made them a part of a larger learning exercise. Thirteen sections of United States History to 1877, offered between the spring semester in 2003 and spring semester 2005 comprised the study group. The experimental group consisting of nine sections wrote essay questions on all the exams in the course. The control group made up of five sections did not write any essay answers. All of their exams consisted of objective questions only. To investigate the influence of essay writing on learning historical material the study administered the same pre- and post-tests to the students of both groups and compared the scores of students on these two tests. The questions used for comparison on the pre- and post-tests came from the multiple-choice portion of the Educational Testing Service's published Advanced Placement Exams. These questions are nationally normed and therefore potentially provide the opportunity to compare the performance of these students with student performance nationally.13 Using these questions also helped to eliminate instructor bias in constructing the multiple-choice questions. The pre-test was a stand-alone exam given on the first day of class. The post-test consisted of some of the same questions embedded into the comprehensive final exam. Only scores on identical questions were compared. |
6
|
|
The hypothesis that motivated the study was that students who are required to write essay questions on exams would perform better on objective post-tests than students whose exams consisted of purely objective questions because essay questions required them to use historical reasoning that, in turn, encouraged students to understand and use historical material, rather than just memorize it. In other words the retention of factual information was linked to the historical reasoning required for the essays. |
7
|
|
All of the students in the various sections of the survey of United States history to 1877 used in this study received the same type of instruction, assignments and assessments, with the exception of the preparation for and the writing of essays on their exams. Course readings included a widely-used United States history textbook, a primary source reader and two book readings. The course included lectures on history content, student-led discussions about their primary-source reader, two essays, written in class, on the book readings, two exams and a final. The lectures emphasized main ideas and significant factual information. Throughout the lectures students were asked to connect ideas presented in lecture to their interpretation of some type of instructional aid such as short, in-class reading, data tables or graphs, or a variety of primary sources. The materials for these inquiry-based activities were usually displayed on a document reader along with a skeleton outline of the lecture. Students were encouraged to fill in the lecture outline as they took notes. They were provided cues about important ideas and historical information with classroom board work. The inquiry-based activities that punctuated each lecture comprised at least fifteen minutes of each fifty-minute lecture period. Once a week students, themselves, conducted the inquiry-based activity when they took turns leading a discussion centered on their primary source reader. |
8
|
|
The two one-hour mid-term exams and the two-hour final consisted only of multiple-choice questions in the non-essay-writing group. The multiple choice questions in some cases asked students to recall historical information and in other cases to apply what they had learned in order to answer a question. The exams for the essay-writing group included essays that required students to develop an argument about a historical topic. One of the essay questions on the final for this group drew upon students' comprehensive knowledge of material from the beginning of the course. |
9
|
|
One week before their exam the group of students who wrote essays received a list of three essay questions any of which could appear on their exam. The design of the essay questions, together with the recommendation that students outline the questions in advance of the exam were intended to encourage students to analyze, and to synthesize information they learned in class lectures and discussions. Except for the final, the essay questions dealt with significant themes from the historical material most recently covered in class. The questions asked students to develop an argument based on their knowledge about these themes or time periods. The questions also required that students combine the factual information from class lectures and readings and to use the analytic skills they practiced in class. Their aim should be to develop a logically reasoned, historically accurate essay in response to the question. Students were advised, in preparing for their exams, to carefully outline their responses to all three of the study questions. They should formulate a thesis statement and sketch basic ideas and arguments in support of their thesis. The instructions for the essay study questions also advised students that their outlines, if thoroughly prepared, comprised important study guides for the exam as a whole. Specifically, the historical information they assembled in support of their arguments for each essay question could appear in other parts of the exam, particularly in multiple-choice questions. Thus, it was important for students to outline all the questions because, even if a particular essay question did not appear on the exam, information used to answer the essay could be on the exam. Diligent preparation for essays thus gave students in this group an advantage even on objective portions of the exams. |
10
|
|
The study group comprised three hundred sixty-five students, who completed both the pre-test and post-test. The total number of students enrolled in the course during the course of the study was five hundred and two students. The thirteen sections of the course that took both the pre-test and the post-test ranged in overall size from thirty-three students to forty-nine students in a section with an average of thirty-nine students per section.14 The number of students who actually took both the pre- and post-test ranged from twenty-one students to thirty-four students per section with an average number of twenty-four students per section taking both tests. The discrepancy between the number of students in the study group who took both the pre- and post-tests, and the number of overall students has two causes. Some students registered late for the class and, therefore were not present on the first day of class when the pre-test was administered and other students who withdrew from the class or stopped coming to the class and did not take the post-test. |
11
|
|
Student's "t-test" (Microsoft Excel) was used to analyze and compare pre-test and post-test performance. Specifically the "t-test" determined whether there were statistically significant gains in performance between the pre-test and post-test. The "t-test" also determined the mean test performance of students in essay writing and non-essay writing sections. This provided a basis to determine whether there were significant gains in student performance between essay writing and non essay-writing sections.15 |
12
|
| |
|
Evaluating Essay Writing on Exams | |
|
The results of the pre- and post-test scores are informative. Pre-test scores were generally consistent between sections with students scoring between thirty and forty percent on the pre-test. The exception to this was fall 2004 class where the pre-test scores averaged between forty and fifty percent. [See Figure I] Post-tests were higher with the exception of the 1:00 p.m. section, spring 2005, (Class 13) that demonstrated no improvement. Post-test scores ranged from five to thirty-three percent higher than pretest scores. [See Figure II] |
13
|
|
|
|
|
|
|
|
Figure III suggests that essay writing on exams made a difference in students' scores, and potentially their understanding, as reflected in the greater net gain of the post-test over pre-test scores for the nine sections in the first three semesters when essay questions were part of the exams. During these semesters average scores increased between twenty and thirty percent. However during the fall 2004 and spring 2005 semester sections, where there was no essay writing on exams, demonstrated a smaller gain of less than fifteen percent. In the case of the 1:00 p.m. section of spring 2005 there was no gain at all. These results support the assertion that essay writing improves history learning; the gain was statistically significant. (P=0.017) |
14
|
|
|
|
|
The two fall 2004 sections of United States history students, without essay writing on exams, scored unusually higher on the pre-test yet had post-test scores similar to previous semesters. On the one hand, one might have expected to find a greater net gain if students had greater prior knowledge of the material but this was not the case. The lower than expected change may been a result of lack of writing. On the other hand, other factors may have formed an upper boundary to student learning. This possibility is suggested by the uniformity of post-test scores for all sections. Pre-test scores from the spring 2005 semester were in line with those of the first four semesters. [Figure I]. Like the fall 2004 semester, the spring 2005 semester students did not have essay writing on exams. If essay writing has a positive effect on student learning then post-test scores should be lower for the spring 2005 section than any previous semester. Actually, the post-test scores for the spring 2005 sections are the lowest for all the sections and they have the smallest net gain. This suggests that the absence of essay writing was also, at least in part, responsible for the lower gain in the fall 2004 |
15
|
|
What do the results from this investigation tell us about the design of a survey course in United States history? This investigation demonstrates that simply adding the writing of essay questions to exams in United States history survey courses has a positive effect on student learning. Specifically, the findings provide quantitative support for the widely held understanding among history teachers that writing is beneficial for content learning in history. Given that the results of this investigation do not specifically address other learning objectives, most importantly critical thinking, analysis and communication skills that are directly related to writing, it is likely that requiring essay writing on exams is an even more effective teaching strategy for these overall earning objectives. Research examining the effect of Writing Across the Curriculum programs with history learning corroborates this assumption.16 Our findings provide quantitative support for the American Historical Association guidelines that are already in place. We think they justify the added effort required on the part of survey course instructors though, of course, reducing class sizes and overall must be the aim of colleges and universities to make it possible to include more writing in history instruction. As historian Lendol Calder observes in his discussion of history teaching and the scholarship of teaching and learning, "If ever a course demanded inquiry into learning surely it would be the introductory history survey."17 |
16
|
|
Notes
Earlier versions of this paper were presented at Faculty Research Days at UCM in March 2005 and at the International Society for the Scholarship of Teaching and Learning Annual Meeting in Vancouver, British Columbia, Canada, in October 2005.
1. American Historical Association, " Statement on Excellent Classroom Teaching of History," <http://www.historians.org/teaching/policy/ExcellentTeaching.html>; Internet: Accessed 24 February, 2006.
2. The literature pertaining to the goals, strategies and outcomes of Writing Across the Curriculum programs is vast. See Chris M. Anson, John E. Schwiebert and Michael M. Williamson, eds. Writing Across the Curriculum: Annotated Bibliography (Westport, Connecticut: Greenwood Press, 1993) and Education Resources Information Center (ERIC) using Writing Across the Curriculum as the subject search; <http://eric.ed.gov>; Internet; accessed 24 February 2005.
3. Robert B. Townsend, "Latest Figures Show Sizeable Increase in History Majors and Bachelor Degrees," Perspectives 42:4 (April 2004) [on-line]; available from <http://www.historians.org/perspectives/issues/2004/>; Internet; accessed 24 February 2006.
4. Ibid.
5. Robert B. Townsend, Teaching the Introductory Survey: Insights from the College Board's AP © Survey Perspectives, 43:6 (September 2005) [on-line] available from <http://www.historians.org/perspectives/issues/2005>; Internet; accessed 24 February 2006.
6. Ernest T. Pascarella and Patrick T. Terenzini, How College Affects Students: Findings and Insights from Twenty Years of Research (San Francisco: Josey-Bass Publishers, 1991), 87.
7. Wilbert J. McKeachie, Teaching Tips: Strategies, Research and Theory for College and University Teachers (Lexington, Mass: D.C. Heath Publishers, 1994), 201.
8. Ernest T. Pascarella and Patrick T. Terenzini, How College Affects Students, 87.
9. For a discussion about minute papers see, for example, K. Patricia Cross and Thomas A. Angelo, Classroom Assessment Techniques, A Handbook for Faculty, Prepared for the National Center for Research to Improve Postsecondary Teaching and Learning (Board of Regents of the University of Michigan, 1988), 147–150.
10. Two useful essays on essay writing as a teaching methodology in history classes are: John F. McClymer and Kenneth J. Moynihan, "The Essay Assignment: A Teaching Device," The History Teacher 10 (May 1977), 359–371 and Ray W. Karras, "Essay Assignment: Write an Historical Argument," The History Teacher 28 (Aug 1995), 495–502.
11. Mark Thompson, "The Effect of a Writing Across the Curriculum Program on Students In An American History Class: Report On An Empirical Study," (Master's Thesis, University of Oklahoma, 1989), 21–40.
12. Thompson, iv.
13. Source: AP U.S. History Released Exam 1996. Copyright © 1996 by the College Board. Reproduced with permission and A.P. U.S. History Released Exam 2001Copyright © 2001 by the College Board. Reproduced with permission. All rights reserved. <http://apcentral.collegeboard.com>.
14. The teaching load throughout this study was 8 courses a year from spring 2003 through spring 2005. Three courses a semester were U.S. history surveys. The number of students in these sections, taken together, averaged 117 students each semester. This compares to the average class size of 153 students for an introductory class in U.S. history cited earlier in the paper. The source for this information does not include information about the average number of sections taught a year. A separate news item concerning the survey above indicates that the average teaching load for a full time history professor is 5 courses year. See Robert Townsend, "The State of the History Department: The 2001–02 AHA Department Survey," Perspectives (April 2004), available from <http://www.historian.org/Perspectives/Issue/2004/>; Internet; accessed 24 February 2006.
15. Richard Runyon and Audrey Haber, Fundamentals of Behavioral Statistics (Reading, Massachusetts: Addison-Wesley Publishing Company, 1976), 160–161.
16. Mark Thompson, "The Effect of A Writing Across the Curriculum Program."
17. The need for research about the college classroom is corroborated in Lendol Calder, "Looking for Learning in the History Survey," Perspectives 40:3 (March 2002), available from <http://www.historian.org/Perpectives/Issues/2002>; Internet; accessed 24 February 2006.
|
Content in the History Cooperative database is intended for personal, noncommercial use only. You may not reproduce, publish, distribute, transmit, participate in the transfer or sale of, modify, create derivative works from, display, or in any way exploit the History Cooperative database in whole or in part without the written permission of the copyright holder.
|