37.4  
Journals link Search link Partners link Information link
August, 2004
Previous
Next
The History Teacher

Table of Contents
List journal issues
Home
Get a printer-friendly version of this page
 
 

Race and Gender Issues on the AP United States History Examination*

Uma Venkateswaran
Educational Testing Service

OVER THE PAST TWO DECADES, we have made remarkable strides in examining, documenting, and incorporating race and gender issues in our history courses, but it is time to take a look at the ways in which these curricular and pedagogical changes have impacted the Advanced Placement United States History Examination. Here we will focus on three interrelated issues: first, the inclusion of race and gender questions on the examination; second, the steps taken to ensure that the exam is fair to all candidates; and finally, the performance of women and minority students on the examination. 1
      Similarities and differences between female and male educational learning styles and achievements is an intriguing and perplexing problem. Several studies have documented that girls have greater verbal ability while boys have stronger quantitative and visual skills.1 Standardized test results indicate that on history examinations women, African Americans, and Hispanic students tend to perform better on questions that do not include quantitative analysis or maps, charts or graphs, while Asian students score higher on questions not dealing with cultural issues. A solid understanding of these group differences is the first step in constructing a fair test. Standardized tests need to evaluate accurately a student's understanding of specific content matter while minimizing the influence of other factors wherever possible. This paper will examine two interrelated issues concerning race and gender equity: First, the steps taken to ensure the fairness of the AP United States history examination, and second, the performances of women, African Americans, Hispanics, and Asians on past AP United States history examinations. I will start with a brief overview of the exam format and content specifications, then describe Educational Testing Service's (ETS) fairness procedures, and finally review data on candidate performances on the exam. 2
   

AP United States History Examination: Content and Format

 
      The AP United States history examination takes three hours and ten minutes and tests knowledge gained in a college introductory survey course. Last year, the exam was administered to approximately 242,000 students from 900 schools worldwide. It has two sections: a multiple-choice section with eighty questions and an essay section. The first long essay is the mandatory Document Based Question. The students write two other short essays, selecting one essay from each of two essay sets. In answering the essay sections of the test, students perform three different tasks: for the Document Based Question, they interpret several primary documents and integrate the information provided in these documents with their prior knowledge to construct a historical argument. For the other essays they write a thematic essay about a specific topic.2 3
      The AP United States history exam reflects the topics covered in a college-level introductory survey course. The content domain for the exam is determined by a periodic curriculum survey of colleges nationwide. The survey solicits information on the material covered in college surveys and textbooks used. Curriculum surveys are performed every four to five years in order to ensure the exam's currency. For instance, in the early 1990's our survey indicated that the exam did not have sufficient coverage of social history. Based on this survey, the exam specifications were revised to include a higher percentage of social history questions (from eighteen to thirty percent). Broader coverage of social history has meant the inclusion of more women's history and African American history questions. 4
      The inclusion of gender and race questions on the exam has required some delicate balancing between the need to test material that a vast number of students are familiar with and the need to introduce new themes and concepts. In a classroom setting, the instructor can introduce new themes and concepts with ease and expect students to demonstrate an understanding of material that has been covered in class. As AP is a standardized test given nationwide, in order to be fair to all students exam questions have to reflect the mainstream teaching of history and also what is covered in most textbooks. Given these constraints, incorporating race and gender related issues on the exam continues to be an interesting challenge. 5
   

AP United States History and ETS Fairness Guidelines

 
      Let us now turn to the second topic in our discussion. How do we ensure that the test is fair to all students? The AP United States history examination is assembled to meet ETS Corporate Standards for Fairness. The guidelines stipulate that all tests must avoid stereotyping and language that is sexist, racist, or otherwise potentially inappropriate or negative with respect to any group. In order to ensure fairness, we follow two complementary processes: the Test Sensitivity Review Process, and the Differential Item Analysis (DIF). The Test Sensitivity Review Process is performed prior to the administration of the examination and consists of a review of the entire test by a trained internal reviewer. The reviewer checks to ensure that questions avoid language that fosters stereotypes or that includes inappropriate terminology, inflammatory material, or a derogatory tone.3 6
      The second process, the Differential Item Analysis (DIF), is conducted after the administration of the exam and involves an analysis of actual candidate performance on questions. DIF identifies test questions that are harder or easier for members of one group than for members of another group. The procedure matches test-takers of equal ability, using the total test score as a criterion and then examines the similarity or lack of similarity of performance by these matched test-takers on each test question. DIF occurs when test-takers of approximately equal knowledge and level of ability perform in substantially different ways on a test question, e.g. when a test question is substantially more difficult for some test-takers than for others of approximately equal knowledge. ETS runs DIF analyses for five focal groups: women, African American, Hispanic, Native American, and Asian American students. Questions that are flagged by the analysis are classified into three categories: low or negligible C-Dif values, moderate C-Dif values, and large C-Dif values. A panel of ETS experts and outside reviewers reviews every question in the moderate and large categories. The outside reviewers are generally women and minorities selected to ensure that different group perspectives are fairly represented. The panel determines whether a question can be retained in the exam or has to be dropped from scoring. The decisions are based on whether a question is accurately reflecting real differences between groups of test takers or whether the question itself somehow produces unfair differences not related to the construct being tested. We have found that questions on maps, charts, and graphs tend to be hard for women and minorities. We have also found that women do well on women's history and cultural questions, and African American and His-panic test-takers tend to do well on questions relating to African Americans and Hispanics, respectively. 7
   

Sample AP United States History Questions on Race and Gender

 
      This section has a few examples from our 2001 published exam (edited slightly) and a brief discussion of student performance on these questions. These questions not only serve as samples of the kinds of gender questions we have on the exam, they also provide feedback on the material with which students are familiar. Let us first look at gender questions.
Sample 1:
The flappers of the 1920's challenged traditional American attitudes about women by supporting
  1. a federal birth control and abortion rights protection law
  2. gender equality in salaries
  3. an equal rights amendment
  4. greater freedom in manner of dress and moral behavior
  5. a federal law to establish prenatal clinics in rural areas

      This question tests students' understanding of shifts in women's role in the early twentieth century. This is a basic identification question and requires little analysis other than an awareness of the major changes occurring during the 1920's. The correct answer is choice D, although all the other choices deal with women's rights issues as well. This question was quite easy and 85% of them answered it correctly. Female students performed better than male students on this question by a margin of 18%.

Sample 2:
Margaret Sanger is best known for her
  1. contribution to the radical suffragist movement
  2. endorsement of coeducation
  3. advocacy of birth control
  4. presidency of the Women's Christian Temperance Union
  5. organization of the Women's Trade Union League

      This question asks about the advent of birth control. The correct answer is choice C. The question requires little analysis other than the correct identification of Margaret Sanger's work. This question was harder than the previous one and only 55% of the students answered it correctly. As this is a basic identification question, the difficulty level is somewhat surprising and could well be a reflection of the fact that this may be material that is not be covered in most classes. Female students performed better than male students by a margin of 13%.

Sample 3:
"The problem lay buried, unspoken, for many years in the minds of American women. It was a strange stirring, a sense of dissatisfaction, a yearning that women suffered...each suburban wife struggled with it alone. As she made the beds, shopped for groceries, matched slipcover material, ate peanut butter sandwiches with her children, chauffeured Cub Scouts and Brownies, lay beside her husband at night—she was afraid to ask even of herself the silent question-'Is this all?'"

The author of the statement above most likely was
  1. Angelina Grimke
  2. Susan B. Anthony
  3. Betty Friedan
  4. Angela Davis
  5. Phyllis Schlafly

      This question requires some analysis as it requires students to read the passage, understand the general thrust, and relate it to one of the choices. The correct answer is C, and this question was somewhat hard because only 39% of the students answered it correctly.

Sample 4:
Which of the following emerged during the Progressive Era as the most influential advocate of full political, economic and social equality for African Americans?
  1. W. E. B. DuBois
  2. Frederick Douglass
  3. Booker T. Washington
  4. Ida B. Wells
  5. Langston Hughes

      This question tests students' understanding of the debates within the Black leadership about how to address the racial inequity of the late nineteenth century. The correct answer is A and the choices are chosen to test student understanding of the positions of various prominent Black figures in this period. 58% of the students answered this correctly, indicating that it was a moderately difficult question. However, African American students performed better than male white students by a margin of 13%.

Sample 5:
All of the following contributed to the growth of the free African American population in the United States in the early nineteenth century EXCEPT
  1. the gradual emancipation laws of individual states
  2. manumission granted for Revolutionary War service
  3. manumission granted by slaveholders' wills
  4. natural increase among free African American people
  5. federal constitutional provisions for emancipation

      This question tests students understanding of the legal basis for slavery and the experiences of free Black people. The correct answer is E. This question was hard, because only 35% of the students answered it correctly. But white male students performed better than African American students by a margin of 46%. This may well be due to the format of the question. Generally, a question that asks the student to select the incorrect answer tends to be very hard.

Sample 6:
The first massive migration of Black American people from the south occurred during which of the following periods?
  1. Immediately following the Civil War
  2. During and immediately after the First World War
  3. During the Great Depression
  4. In the decade after the Second World War
  5. During the civil rights movement of the 1960's

      This question tests students understanding of major changes in migration patterns of African American people; the correct answer is C. Surprisingly, only 31% of the students answered this correctly; it was a hard question. African American students performed better than white males by a margin of 36%.
 		8
      We have made efforts to include more race and gender-related topics in the essay section as well. Our 1981 and 1997 document based questions dealt with women's history issues and our 1995 DBQ was an African American history question. We have other short essay questions that deal with race and gender issues as well. 9
      For example, here is a question from our 1990 exam:
From the 1840's through the 1890's, women's activities in the intellectual, social, economic and political spheres effectively challenged traditional attitudes about women's place in society. Assess the validity of this statement.
 		10
      Interestingly, of about 90,000 students who took the exam, only, 7700 students selected this essay. Half of these were female and half male. Here is another question from our 2000 exam:
Discuss with respect to TWO of the following the view that the 1960's represented a period of profound cultural change:
  • Education
  • Gender roles
  • Music
  • Race Relations
 		11
      This question was selected by 80,000 female and 58,000 male students, which represented 80% of the total number of students taking the exam. This question was popular among minorities as well, as 14,334 Asian students (which is 66%), 7,469 African American students (which is 82%), and 9,765 Hispanic students (which is 77%) chose this essay. As most students selected gender roles and race relations as the choices, this question illustrates that students are fairly comfortable with these issues and topics, a remarkable change from our 1990 data. 12
      To summarize briefly, we have recently included more African American and women's history questions on the exam and in general the feedback in terms of students' performances has been reassuring. By and large, students performed well on these questions, an indication that they are familiar with these issues and had no difficulty dealing with them on the exam. 13
   

Performance of Women and Minorities on the AP US History Examination

 
      Now let us turn to a discussion of how women and minorities actually perform on the examination. A quick survey of the comparative performance on several years of AP exams for these groups reveals that the different skills measured by the multiple-choice section, the document based question, and the thematic questions result in different performances across groups. While this data is somewhat controversial, it provides critical feedback regarding group differences in learning styles, and may be relevant to in-class experiences of teachers. Table 1 shows the differences in the average scores of male and female, African American, Hispanic, and Asian students on the multiple-choice and free-response exam sections for the past eleven years. The maximum score in both sections is ninety points. The results of this data are striking. First, male students consistently outperform female students on the multiple-choice section. These data are consistent with the findings of a nationwide gender study conducted at ETS in 1997.4 The study indicated that there are small, yet consistent gender differences in all social science subjects. Male students scored higher than female students in history, economics, and geography. Female students scored higher on the verbal and literature tests. In history, the gap has narrowed somewhat since the test specifications were revised to include more social history questions. This is most likely due to the fact that male students prefer traditional political and diplomatic history while female students relate better to and score higher on social history questions. 14
      There is also a gap in performance on the multiple-choice questions between males in general on the one hand and sub-groups of African American and Hispanic students on the other. Male students consistently do better than African American and Hispanic students. As Table 1 indicates, the gap between these groups is wider than the gap between female and male performances. In the early 1990's, there was a ten-point gap between African American and white students and a six-point gap between Hispanic and white students. This gap seems to have widened in the last couple of years to thirteen points for African American students and fifteen points for Hispanic students while the difference for female students is around five points.. This is likely a result of a significant increase in access to AP by women and minority students. Several states have passed legislation providing funds to establish AP programs and also mandate student participation. As Table 3 shows, there were huge increases in the number of women and minorities taking the exam over the last ten years. The increases were considerably sharper over the last couple of years. The percentage increase between 2000 and 2002 was as follows: forty-four percent for female, forty-one percent for African American, fifty-eight percent for Hispanic and twenty-one percent for Asian students. These increases combined with declining performances may indicate schools need to do more to prepare students for the challenges of the AP examination. 15
      The performance of Asian students on the examination contrasts sharply with the performance of other minority groups. Asian students consistently outperformed white students on the multiple-choice section. This may suggest that the performance gaps between white students and African Americans and Hispanics is not primarily due to cultural diversity but a question of the course taken and how well students are prepared for the examination. The growth of the AP program is quite laudable in the ever-expanding opportunities it presents students; however it also underscores the need to focus attention on increasing teacher resources and strengthening teacher preparation. As Mike Johanek has noted, "balancing mission and market in educational endeavor" is indeed an ongoing challenge.5 16
      The widening gaps in the performances of different groups over the last couple of years is very disconcerting and there seems to be no clear explanation at this point. The content of the exam has not shifted over the last two years. We looked at how female, African American, Hispanic and Asian students performed on colonial, nineteenth-century, and twentieth-century questions and found no particular differences across chronological areas. We also looked at how these groups performed on political, diplomatic, social, economic and cultural questions and found very little difference, except for 2001 when female students found diplomatic history questions slightly harder than political, social, economic, and cultural questions. 17
      Women and minorities consistently performed better on the essay questions than the multiple-choice section. Although for most of the years presented, male students continued to do better on the essay section as well, the differences are much smaller. Because of the different skills needed for performing well on the DBQ, we see in Table 2 that the differences between female and male performance is almost negligible. The differences are somewhat larger for African American and Hispanic students (ranging from three tenths to eight tenths) but the gap is considerably narrower than the one for the multiple-choice section. Asian students outperformed white students on the essay section, indicating that preparation rather than diversity may account for differences in performance among these groups. 18
      Having said that, there are a few interesting data points here that indicate a correlation between topics and candidate performance. African American and Hispanic students performed the best in 1995, the year the DBQ topic was on Reconstruction, a topic from African American history. Females outperformed males during the years 1993, 1997, 1999 and 2002. These were years when the DBQ topic related to a social issue. Predictably, this difference was largest in 1997, when we had a question on women's history. Male students on the other hand, had a stronger performance in 1994 and 2001, both years when the DBQ dealt with issues in diplomatic history. These findings confirm our earlier observations that female students tend to do well on social history, while male students are more comfortable with political and diplomatic themes. 19
      These conclusions are further borne out when we look at choices in the essay section. As noted earlier, students can select one out of two essays in parts B and C of the exam. I reviewed the choice data over the last ten years. For most of the years, all groups had similar percentages of students selecting each question. However, in 1994 there were some interesting differences. Seventy-two percent of female, seventy-four percent of African American, seventy-two percent of Hispanic and sixty-nine percent of Asian students chose question 2 while seventy-two percent of males chose question 3. Question 2, was a social history question on the Puritans that tested understanding of religious movements in the colonial period. Question 3, was a question on foreign policy in the 1790's that tested knowledge of politics and diplomacy in the late eighteenth century. 20
      To conclude, we have taken measures to ensure racial and gender equity and fairness on the AP United States history examination, but we still have a long way to go. We have revised the contents of the exam to include more social history and, thereby, more race and gender- related questions. We also ensure that the tests have as little race or gender bias as possible through our fairness reviews and DIF analyses. However, the results indicate a persistent gap in the performances of white males vs women and minorities on the multiple-choice sections. While this issue needs further exploration, it looks as if the gap reflects the democratization of the exam as the number of women and minority candidates has risen sharply in recent years. There is much work to be done, but it is clear that we have learned a great deal over the last decade about what is required to design tests that give students a fair chance to demonstrate their understanding of the American past. Concerns about gender equity and diversity have transformed the AP exam over the last decade, even as issues of gender, race, and ethnicity have had a major impact on the ways in which we teach American history. 21



Table 1:
Differences in Performances on multiple-choice questions
Year Mean Differences
Between Female
and Male
Performance		 Mean Differences
Between African
American and White
Performance		 Mean Differences
Between Hispanic
and White
Performance		 Mean Differences
Between Asian
and White
Performance
1992 5.27 10.32 6.69 -1.3
1993 5.01 10.27 6.19 -1.4
1994 4.66 9.27 5.26 -1.1
1995 4.76 10.88 7.38 -1.5
1996 5.21 9.76 8.04 -.30
1997 4.39 11.07 8.4 -0.95
1998 4.12 10.58 10.28 -1.33
1999 4.58 11.95 11.15 +.11
2000 4.0 11.55 9.96 +.56
2001 5.27 13.14 15.37 -0.8
2002 5.29 13.54 15.58 -0.67
 



Table 2:
Performance on the Document Based Question
Year Mean Differences
Between Female
and Male
Performance		 Mean Differences
Between African
American and White
Performance		 Mean Differences
Between Hispanic
and White
Performance		 Mean Differences
Between Asian
and White
Performance
1992 .01 0.61 0.34 -0.11
1993 -.01 0.67 0.39 -0.12
1994 .11 0.82 0.38 -0.18
1995 .01 0.30 0.32 -0.08
1996 .01 0.73 0.57 -0.14
1997 -.25 0.67 0.37 -0.10
1998 .05 0.84 0.55 -0.18
1999 -.05 0.86 0.60 -0.07
2000 .04 0.84 0.64 -0.04
2001 .13 0.85 0.69 -0.02
2002 -.15 0.84 0.76 -0.04
 



Table 3:
Percentage Increase in Women and Minority Candidates Base Year is 1992
Year Female African
American		 Hispanic Asian White Male
1995 31% 27% 39% 21% 20% 36%
2000 86% 115% 168% 81% 73% 67%
2001 102% 144% 214% 90% 87% 81%
2002 124% 162% 266% 98% 105% 98%
 


Notes

*.Ê College Board, Advanced Placement Program, AP, and AP Central are registered trademarks of the College Entrance Examination Board.

1.Ê For a discussion of the literature on this subject see, Nancy Cole, ETS Gender Study: How Females and Males Perform in Educational settings, ETS, 1997.

2.Ê Eric Rothschild, "The Impact of the Document Based Question on the Teaching of the History of US", in The History Teacher, Vol 33, No 44, August 2000, p. 495-500; Alden T. Vaughn, Grading the AP exams in American History, AP, College Board, 1983

3.Ê Overview, ETS Fairness Review, ETS, 1998.

4.Ê Nancy Cole, ETS Gender Study, p. 13-15

5.Ê Michael Johanek, "A Faithful Mirror, Reflections on the College Board and Education In America" p.xxiii (College Board, 2001)

Content in the History Cooperative database is intended for personal, noncommercial use only. You may not reproduce, publish, distribute, transmit, participate in the transfer or sale of, modify, create derivative works from, display, or in any way exploit the History Cooperative database in whole or in part without the written permission of the copyright holder.

 



August, 2004 Previous Table of Contents Next