About a year ago, Freddie deBoer wrote what he called the ‘progressive case‘ for the SAT. Mostly he used the SAT as a convenient stand-in for standardized testing in its American form.

DeBoer’s take on this was provocative and surprising. He took some flak. The general “left” line has been against standardized testing. And it has become one of several points at which liberals and leftists depart: the liberal as the technocratic tester, set against the leftist as the advocate for a free and democratic classroom sans test.

I find this all rather oversimplified. Here I’ll evaluate both deBoer’s argument in favor of a ‘progressive’ view of standardized testing and leftist arguments in favor of a ‘regressive’ view of standardized testing.

I find both arguments lacking.

DeBoer’s ‘Progressive Case’ for Standardized Testing

You can read DeBoer’s full argument at Jacobin. Here’s my summary of what Freddie claims:

1. Test development has processes and measures in place to prevent bias.

2. Any bias that survives the test development process isn’t unique to standardized testing. It’s a deeper feature of society.

3. The only real alternative to testing (i.e., ‘Holistic Assessment’ through GPA, activities, etc.) is more biased than standardized testing.

4. Due to (1) – (3), test scores are the best route for students from marginalized groups to gain admission to good colleges.

5. Therefore, ‘progressives’ should support standardized testing, at least in the short- to mid-term.

What DeBoer Gets Right

First, let me acknowledge that Freddie gets some things right here. (1) and (2) are correct in part. But I’ll say more about those below.

(3) is almost certainly correct. ‘Holistic Assessment’ of GPA and outside activities, used to the exclusion of standardized tests, will almost certainly benefit privileged students and advance inequality.

Privileged students have far more opportunity to achieve in these areas than other students do. On this topic, I’d recommend Dream Hoarders by Richard V. Reeves. Reeves’s point is that privileged students are better able to find and take advantage of these sorts of opportunities.

And so, if anything, Freddie may soft-pedal his case for (3). Colleges don’t treat all high school GPAs the same. More competitive colleges are less likely to admit a student with a 3.9 GPA from a “weak” school than a student with a 3.9 GPA from a “strong” school. Standardized testing provides an avenue for the former student to overcome this bias.

Problems with DeBoer’s Argument

The reason Freddie’s argument doesn’t add up to a ‘progressive case’ is that he overplays his hand, especially at (1) and (2).

On (1), he talks a bit about differential item functioning (DIF), which is a statistical method testing companies use to ensure that test questions (items) and tests (forms) aren’t biased against members of certain groups. So far, so good. But not all companies use DIF.  Some companies use it on all their tests, others on only some of them. It’s a good tool that, in my view, companies should use universally and consistently.

On (2), Freddie’s correct that remaining biases are structural, but that doesn’t necessarily let the testing industry off the hook. Even if biases in testing are repeated elsewhere, the mere repetition of bias can have a cumulative biased impact. Those of us concerned about bias need to look at both single-case and cumulative bias.

From these considerations, I think deBoer fails to make his progressive case for standardized testing. Testing isn’t bad, as I’ll argue below. But it’s not a big net good, either. Whether and when tests are good or bad depend on many factors.

Arguments Against Testing

Leftist arguments against standardized testing come in three basic forms. I’ll sketch those out here:

1. The Common Core State Standards (CCSS) and “Common Core tests” are taking time away from classroom instruction. It’s part of a deeper agenda to undermine teachers’ unions, make classrooms “teacher proof”, and set aside the educational mission of schools in favor of a career skills-focused agenda. Therefore, we should move away from testing in favor of things like smaller classroom sizes, better services for students, and better teacher pay.

2. Standardized tests originated from IQ testing, which is a biased Western construct grounded in imperialism and racism. We can’t divorce testing from this legacy and should therefore eliminate it.

3. Tests are biased against some groups, particularly against people of certain races or socioeconomic (SES) levels. Therefore, we should reject testing in favor of criteria like GPA and outside activities.

To be clear, I put these three not in order of prevalence, but in the order I’d like to respond to them. I think (3) is probably the argument I’ve seen most often, followed perhaps by (1) and then (2).

But I think all three arguments are pretty bad.

Response #1a: Common Core and Testing

The first argument conflates several distinct problems, and we can dismiss it quickly.

The CCSS are a set of educational standards, not a test. As such, there’s no such thing as a “Common Core test.” Companies can write tests that assess what’s in the CCSS, but many of those tests existed well before the CCSS did. And they will continue to exist, even if we get rid of the CCSS. These issues go much deeper than standardized testing, which is something that will likely be around in some form no matter how we address the structural issues with the education system.

And so, we can dismiss this argument as pretty bad.

Response #1b: Common Core and Testing

As to the substance of the claims, I think there’s a case to be made for many of them. Common Core addresses only English and Math. It doesn’t address, for example, Science and Humanities. It downplays literature in favor of something called “informational writing,” which gives a certain nod to ‘career skills’ at the expense of the humanities.

In the wrong hands, I think this narrows one’s education and reduces quality.

More broadly, I think the critics are right that the education system has put too much power in the hands of school officials, shown too little trust in teachers, underpaid teachers, become too concerned with quantitative assessment of teacher quality, and required students to spend too much time on standardized testing. Within the testing focus, I think schools would do best to move away from norm-referenced tests and move toward criterion-referenced tests. But relatively short and useful college admissions tests are compatible with all of these recommendations.

Response #2: Testing and IQ

I think a lot of leftists endorse the ‘Testing and IQ’ argument. I see it a lot in glib Twitter posts, which doesn’t concern me much. But better figures and outlets, such as Ibram X. Kendi in Stamped from the Beginning, also endorse it.

I find this surprising because in its simplest form, the argument is a textbook genetic fallacy. To argue that tests are regressive just because they have regressive origins is to argue on irrelevant grounds. The argument works only if tests retain these origins today. That’s largely the issue in (3), but for now I’ll say that the origin story here is of mixed accuracy, at best.

The SAT is the test people usually cite, and it does indeed have origins in IQ. Carl Brigham, an early leader and promoter of the SAT, sat on the advisory council of the American Eugenics Society and promoted IQ. The SAT claimed to measure “aptitude,” which is more or less another way of saying IQ.

However, the ACT is a test taken by more students than the SAT. University of Iowa professor E. F. Lindquist started the test in 1959, and he did so precisely to drop all notions of ‘aptitude’ or IQ in favor of a focus on academic achievement. Since that time, The College Board, which owns the SAT, has revised the SAT in order to drop ‘aptitude’ and remake the test more in the image of the ACT.

The relative success of these two tests at distancing themselves from IQ is a separate issue. But studies generally show that while IQ and test scores are correlated, they’re not the same. And so this argument really does appear to be a genetic fallacy.

Response #3: Testing and Bias

Are the tests biased? What’s difficult about it is that there’s bias and there’s bias. When people say a test is biased, they could mean one of two things:

Bias A: Students who are members of certain groups (e.g., white, wealthier) do better on the test than students who are members of other groups (e.g., black, Latinx, poorer).

Bias B: Students who are members of certain groups (e.g., white, wealthier) do better on the test than students who are members of other groups (e.g., black, Latinx, poorer) relative to what we would expect given the other things we know about them (e.g., GPA, quality of education, coursework, etc.).

These aren’t the same. Bias A is real and it’s a problem, but it’s probably not a testing problem. Whiter and wealthier students are more likely to receive a good education at a good school, get tracked into rigorous courses and ‘college-bound’ programs, and have a higher GPA. If they do better on tests, that’s evidence the education system is biased. The solution is to improve the system and fight its biases, not to kill the messenger.

Bias B, on the other hand, is a big problem for testing if it’s present. But the research shows that it generally isn’t. Black and Latinx students, in particular, do as well on tests as predicted, based on other factors. In fact, black and Latinx students do better on the ACT than their later college GPAs and graduation rates suggest. And so, there’s not only bias in the K-12 education system, there’s also bias in the college education system. Test scores show that black and Latinx students should be doing better in college than they actually do, possibly due to professors’ biases, racism from white students, lack of adequate support, etc.

A Caveat

And yet, our conclusions on whether tests are progressive or regressive are only as good as our evidence. While the evidence shows that testing over predicts the scores of black and Latinx test takers, the evidence for other groups is less clear.

In addition to race, we might speculate that tests are biased against members of lower SES levels, English language learners (ELL), LGBTQ students, et al. How we divide and study groups, therefore, matters quite a bit. It stands to reason that ELL students, in particular, are a group that might have difficulty with standardized testing in its current form.

A Conclusion: Standardized Testing is Neither Progressive Nor Regressive

I have to conclude by saying, as above, that Freddie deBoer failed to make his case in favor of testing. The benefits are somewhat uncertain.

But opponents of testing fail to make their case, too. Their arguments suffer from logical fallacies, mistaken claims, et al.

I think testing can be, and has been, a lot of things. It’s a tool. If you put together a test using proper methods, and use it for its proper purposes, it’s fine. And even carries potential to correct education system biases. If you use testing to do something it wasn’t designed to do, or if you don’t use the proper test development methods to eliminate bias, or if you test students too often, it’s a problem.

Note: I’m an employee at an assessment company (ACT, Inc.) that produces one of the standardized tests I discuss in this post (the ACT). I work in test development, but my work relates to a different test the company produces, not to the ACT test itself. I’m including the note for a couple of reasons. One, in this post I state my own views, not the company’s views. Thus, the required language: “The personal opinions expressed do not necessarily represent the views and opinions of ACT, Inc.” That’s probably pretty obvious to most of you, since I work as a regular employee, not as an executive or leader who speaks on behalf of the company. But companies have this sort of required language for good reasons. Two, I’ll leave it up to the reader to decide whether this fact biases my views in any way. I don’t think it does, but it’s always a possibility.

Image Source: https://www.flickr.com/photos/thedavisblog/2230010178