Fred Smith Explains Field Tests

I was reading through your site (Change the Stakes) and could not easily locate any info about any past or pending NYS field testing for the elementary or middle grades. I am a Long Island parent whose child has refused to take the NYS ELA and Math assessments this year. We are curious if any field testing is coming next, which I think there is some this year, perhaps even computer based. Any insight you can provide is most helpful. Thanks! --- Parent

There is another kind of field testing — known as stand-alone field testing — coming to our students soon. This is where the field test items are contained in separate booklets and results don’t count except for the publisher; the sole purpose is to try out more items for possible use. This will occur in the first week of June. There are three problems about field testing children in June. First, they know the tests don’t count. Second, it’s June. Third, after a test-heavy school year, it is unlikely they will give their best effort and perform optimally. So, the results derived from the stand-alone field tests will be a misleading basis for selecting items for next year’s exams. ---- Fred Smith: excerpt, Field Tests: Unfair Burden on Students at Schoolbook

As the one who monitors comments at CTS, I can't tell you how many parent requests for assistance on opting out have come in and the CTS parent advice crew have been busy. The hope is that CTS expands its outreach to create a bigger opt out movement on the field tests coming up. Field tests are used to check out future questions and don't count. These questions were embedded in the current tests but there are stand-alone tests coming that count for nothing and amount to child and teacher abuse.

In May 2008, middle school teacher Doug Avella discussed the field test with his classes and they all boycotted the tests. Non-tenured Doug was quickly disappeared from the system as the UFT went, "cluck, cluck" with disapproving nods even though the kids took action based on a fair discussion and never implicated Doug's words. The kids were treated like criminals. Here is a link at ed notes:

Ed Notes Online: Support for Doug Avella Builds

May 27, 2008

The students of I.S. 318 stood up for what they thought was right. They have been taught by a beloved teacher whose job is now in jeopardy. It is critical that we stand up right now and show our support for Doug Avella .... Letter from

Sam Coleman and Geoffrey Enriquez, on behalf of NYCORE
Priscilla Gonzalez and Donna Nevel, on behalf of Center for Immigrant Families
Jane Hirschmann, on behalf of Time Out From Testing
Sally Lee, on behalf of Teachers Unite

Fred Smith is the expert on the issue and sent this response to the parent above.

Your question provides and opportunity to talk about the field tersting.

There are two kinds of field tests (FT)--One kind took place today in ELA in grades 3-8--where FT items were embedded in the test booklets among the other items. The FT items do not count in scoring the test. They are there to be tried out so the publisher can see how the items work and can decide which ones to use next year. The other items will count. They are referred to as operational items. Scores will be based on student performance on these items.

For example, today's 3rd grade ELA test (Booklet 1) contains 30 items (5 reading passages followed by multiple-choice items). Children have 70 minutes to complete the test. They don't know which items count and which do not (i.e., the FT items). The test publisher hasn't said how many of the 30 items are FT test items--only that FT items are included on the test.

A safe assumption is that at least one reading passage about 500-600 words in length was being tried out along with the set of multiple-choice items associated with the passage. If everything was evenly balanced, that would mean about six items were being tried out with this passage--and would take about 14 minutes to complete -- one fifth of the allotted testing time.

The publisher (Pearson) is using four forms of the test to try out items. What does this mean? The four forms will consist of the same operational items, but each form will contain one of four sets of FT items.
So Booklet 1 will have 30 items with the same items that count and a unique set of unrefined items that are being tried out. The publisher represents the four forms as A, B, C and D.

The following is how this arrangement might look, based on the 3rd grade example I gave: (P=operational passage; O=operational multiple-choice items following the passage; FTP=field test passage; FT=field test multiple-choice item)

Form A: P-OOOOOO / P-OOOOOO / P-OOOOOO / P-OOOOOO / FTP-FTFTFTFTFTFT
Form B: P-OOOOOO / P-OOOOOO / P-OOOOOO / FTP-FTFTFTFTFTFT / P-OOOOOO
Form C: P-OOOOOO / FTP-FTFTFTFTFTFT / P-OOOOOO / P-OOOOOO / P-OOOOOO
Form D: FTP-FTFTFTFTFTFT / P-OOOOOO / P-OOOOOO / P-OOOOOO / P-OOOOOO

The State Education Department (SED) has said that the FT material will be interspersed with the operational material. Since students don't know which readings and items will count from the material that is being field tested, students will be expending energy of parts of the test that will probably wear them out and frustrate them, because SED has called for tougher, more challenging items. That will have a bearing on how well they do on the test. It is likely they would do better if the test was limited to the 4 operational passages and 24 operational items.

Furthermore, since one-quarter of the students get one of the four forms, then the children who get Form D or Form C will run into the FT items (by definition less refined than the other items) right from or near the start of the test and have a greater chance of being worn out and defeated before they get to the end of the test compared to children taking Form A--who, even if they don't finish the FT items, will have a better chance of completing all the items that count.

[This has implications in New York City, where the Department of Education (DOE) makes extrapolations based on student test scores to student promotions. teacher evaluations and school ratings--clearly, high-stakes decisions that may well rely on which FT form a given class of students happens to take.]

By the way -- tomorrow ELA Booklet 2 will contain embedded multiple-choice items in all grades--3 through 8. And next week, the first two days of the math tests--Booklets 1 and 2 will contain the FT items.

The other kind of field testing is known as stand-alone field testing. This is where the FT items are contained in separate booklets and the results don't count except for the publisher. The sole purpose is to try out more items for possible future use. This will take place in the first week of June (the same as it did last year.)

SED has never established that taking the field tests is mandatory. Nor has SED cited any legal authority that requires students to sit for either the embedded items or the stand-alone. This lack of authority makes the June stand-alone field test an ideal target for resistance. I believe that not taking this week's and next week's exams on the grounds that children are being forced to engage in field testing would add a strong argument to the reasons they should not have to sit for the exams. More on that below.

Last year 20 forms (12 ELA and 8 math) were tried out in June--a service deliverable not called for in SED's contract with Pearson. The reason for this was that the four April forms did not generate enough of a pool of items to build this year's (i.e., this week's) tests. So the June 2012 field testing was hastily arranged.

There are three things wrong with field testing kids in June. First they know the tests don't count for anything. Second, it's June. C'mon, after a year of being bashed by tests--you're asking kids to concentrate on more items, take them seriously and perform optimally (the rationale behind embedded field testing). Third, as a consequence, the children are not motivated to perform their best. Therefore, the results derived from the June stand-alone field tests will be a misleading basis (or source of intelligence, if you will) for selecting items for next year's exams.

But there is a more fundamental and insidious problem lurking in the field tests. It is a reflection of the top down arrogance in Albany that insinuates children into taking the field tests (both embedded and stand-alone), which are nothing more than experimental, serving the commercial interests of the test publisher, and making children the subjects of research -- without bothering to seek the permission of parents to allow their children to participate in otherwise un-required testing activity. Doing so violates the right of parents to do what is best for their children. It is a form of exploitation to perpetuate the kind of high-stakes testing that has channeled all the creative energies of teachers and students into testing. Parents are entitled to know what is going on and to have a say in what's going on.

Making matters worse there will be stand-alone field testing again this year at the beginning of June--throughout the state in ELA and math. Most schools will have assignments to administer the try-out exams. And, just as SED did last year--it will try to keep parents in the dark about the field tests until the last minute. That's because SED understands well that these tests are not required and, if parents knew they were coming and more classroom time was being sacrificed, they would opt out of the stand-alone field tests. SED obviously feels there is a greater need to satisfy the publisher and keep the testing going than to show respect and regard for parents.

The answer going forward next week and up to the first week of June is to mobilize parents to refuse to take tests with the embedded FT items, which parents never signed onto. And between now and June --it is imperative to mobilize against the coming stand-alone field tests and to reach consensus to boycott them. This is a fight we can and must win to prevent the seeds of the next year's test from sprouting.

Fred Smith