|
|
|
Bad Tests, Good Outcomes: Using State Teacher Licensing Scores to Improve the Curriculum, Help Students Meet Their Professional Goals, and Keep Relationships Together
Russell Olwell
Eastern Michigan University, Ypsilanti
| AS UNIVERSITIES BECOME MORE INVESTED in assessment and outcomes, department chairs and deans increasingly are asked to figure out why some departments score well on tests, and others dismally low. At all NCATE accredited colleges and universities, the pressure is on to reach the 80% pass rate threshold, with fear that any scores below that will raise questions as to the rigor of the program. |
1
|
|
The tests that our students take in order to become teachers vary widely in quality—they include the Educational Testing Service's Praxis II test (used in 44 states), as well as tests designed by states individually (often through a large provider such as National Evaluation systems). Most historians would not find any one day multiple choice history test a valid measurement of historical thinking skills, never mind a good way to decide who gets to teach in a classroom for the next three or more decades. |
2
|
|
Critics of standardized testing, such as Alfie Kohn and Ann Lewis, simply find testing for teaching skill invalid in itself, as no paper and pencil test can capture great teaching (Lewis 2005, Appleman and Thompson 2002). Others in the field point to the disproportional failure rates among members of minority groups and test takers from lower economic groups (Wakefield 2003, Albers 2002, Tellez 2003). Statistical studies of teacher testing point to critical thinking skills as the best predictor of test taking success, not necessarily the content knowledge the test is supposed to be measuring (Chambers, et al. 1999). |
3
|
|
Educational psychologist Lloyd Bond has developed a schema for thinking about testing and how to help students. In his two-by-two diagram, there are four groups we need to account for. There is one group of students who are both have the skills to pass the test, as well as the skills to teach. Another group is not able to teach, but is able to pass the test. Members of the third group are able to teach, but not pass the test, and a final group is able to do neither. Departmental efforts need to focus on the third group—those who simply need help in test-taking skills or content holes. Advising efforts need to help those in the fourth group out of their programs and to identify areas they have talents in (for a truly excellent summary, see Bond, 2006). |
4
|
|
However, teacher test scores need to be only one part of a longer conversation about how departments assess student learning, especially those entering K-12 teaching. This article will set out some guidelines and processes for examining test score data, in order to save time for faculty members and department heads new to the process, drawing from five years of attempts by one history department to improve performance on state teacher licensing exams. While tests cannot answer all the questions about what our students know, they can often reveal the blind spots in our curriculum, and the concepts that students are just not getting. It is important to find the key information in test system, if only to ensure the survival of the department's teaching program. |
5
|
| |
|
Step 1: Identifying Test Takers | |
|
Who is taking the test in your institution's name? The answer to this question might seem obvious, but it is not. In Michigan, for many years, anyone could sign up for the Michigan Test of Teacher Certification (MTTC) test, mark that they were a graduate of a university, and the score would count for or against that university—no questions asked. |
6
|
|
As a result of complaints by universities, the state changed this policy. Now, universities have a week to examine the names of people signing up to take the exam, and they can indicate those who are bona fide students or graduates, defined as individuals who have completed 90% of the program in which they are being tested. As a result of this policy change, our department can examine the names of students signed up to take the exam stating that they are from Eastern Michican University (EMU), and eliminate those who have not been our students, or those who are not yet far along enough to qualify. |
7
|
|
This act of looking up student transcripts for eligibility has another healthy effect. Looking at lists of students and what classes they have taken can be a sobering experience. Coming across transcripts full of failures, incomplete, and withdrawals is a reminder for the department that there are students not making their way through their major successfully, and who need to be identified and counseled out of teaching. |
8
|
| |
|
Step 2: Curriculum Analysis | |
|
Once initial screening has been done, the next thing to analyze is whether students' grades in departmental classes align in pass rates. In other words, do students who take all of the required classes and receive high grades pass the state test? If not, the problem may very well be a curriculum that is out of joint with the state exam and probably the state curriculum. |
9
|
|
In an analysis of grades and test scores, the department found that many test failures were successful in survey level coursework, but faltered in upper-level classes. This was a clear sign, as Chambers (1999) had argued, that lack of critical thinking skills was hurting these students both in upper-level course work and on the MTTC test. This finding also pressed the department to build more critical thinking skills into survey classes, to better communicate to prospective majors what thinking skills were needed to be successful in the discipline. |
10
|
|
Curriculum analysis can also highlight gaps. The widest gap usually was found in survey-level classes, where students needed to take only one-half of the U.S. survey sequence (1492 to 1865 or 1865 to the present), and two-thirds of a three-class western civilization survey sequence. This curriculum did not at all align with the state requirements, which stressed world (rather than European) history and covered all of U.S. history. |
11
|
|
While the department did not embrace changing the sequences, it did move to a major and minor that required a full sequence in U.S. and world history. This change will also better prepare students not only for the MTTC, but also to teach in schools, many of which have globalized their own curricula more fully than colleges and universities. |
12
|
| |
|
Step 3: Coaching Students to Succeed | |
|
Students in Michigan can take the state teaching test as often as they like. Thus, the one student who failed the MTTC over ten times counted against us each time. It pays to get in contact with failing students and to have resources available for them. |
13
|
|
For some students, test anxiety alone may have kept them from passing. The threat of not being able to pursue a career, the stigma of failure, and hints from spouses and partners that test failure may result in the ending of a relationship can all bring students into testing despair. Having some practice can help students reduce their anxieties and increase their scores to match their knowledge. |
14
|
|
Stereotype threat can also derail students on high stakes exams. Rigorous psychological studies have shown that if students, particularly minority students, are faced with a high stakes test, they score lower due to their belief that they will not score at their level of knowledge (Steele, 1999). |
15
|
|
At first, the department offered in-person review sessions. However, these appeared to only help students who would pass the test anyhow. Two years ago, EMU continuing education created an online test review, which consists of resources (mostly online) for each period of history and a few hundred questions to give students a chance to practice in each area. The ability to practice a wide variety of questions, each with linked internet resources to further learn about that topic, gives students with test anxiety a chance to practice and make the process of taking the test less terrifying. |
16
|
| |
|
Step 4: Get Involved in the State Testing System | |
|
Being proactive in the teacher-testing arena is important, as the decisions made by state-level committees can have a lasting impact on your department. It is important to know the names and faces of the people at the state level who coordinate testing and accreditation. They can help you with data and appoint you to key committees that review and create tests. Being outside the system is what causes many faculty to foam at the mouth about state tests. Once faculty members know who is supervising testing, and those officials know of your availability to serve on state committees and task forces, faculty will have a person, and not a faceless bureaucracy, to contact. |
17
|
| |
|
Step 5: Measure Your Results | |
Test scores at EMU have gone up as a result of these efforts. In 2000, our history scores averaged a pass rate of 58%; by 2008, that was up to an average of 92%, higher than our NCATE goal of 80% and the state average of 78%. However, the incremental changes we have made have given us some tools to address the situation and some measure of success. Most importantly, we are moving forward knowing that we have done our best to help our students meet their professional goals and provide continuing support to those in need.
References
|
18
|
Albers, P. 2002. "Praxis II and African American Teacher Candidates (Or, is Everything Black Bad?)." English Education 34 (2002): 105–125.
Appleman, D. and M. Thompson. "Fighting the Toxic Status Quo." English Education 34 (2002): 95–104.
Bond, Lloyd. "My Child Doesn't Test Well." Carnegie Perspectives, http://www.carnegiefoundation.org/perspectives/sub.asp?key=245&subkey=2369.
Chambers, S, et al. "Predictors of the Success on the Texas State Certification Tests for Secondary Teaching." College Student Journal 33 (1999):1–6.
Lewis, A. "Highly Qualified Teachers: We Know 'Em When We See 'Em". Phi Delta Kappan 86 (2005): 563–564.
Steele, Claude. "Thin Ice: Stereotype Threat and Black College Students." The Atlantic Monthly 284, no. 2 (August 1999): 44–54.
Tellez, K. "Three Themes on Standards in Teacher Education: Legislative Expediency, the Role of External Review, and Test Bias in the Assessment of Pedagogical Knowledge." Teacher Education Quarterly 30 (2003): 9–18.
Wakefield, D. "Screening Teacher Candidates: Problems with High-Stakes Testing." The Educational Forum 67 (2003): 380–288.
Zigo D. and M. Moore. "Accountability and Teacher Education: Just Like Everyone Else—Teaching to the Test?" English Education 34 (2002): 137–155.
|
|
|
Content in the History Cooperative database is intended for personal, noncommercial use only. You may not reproduce, publish, distribute, transmit, participate in the transfer or sale of, modify, create derivative works from, display, or in any way exploit the History Cooperative database in whole or in part without the written permission of the copyright holder.
|