September, 2003
The Meeting in Brief:
The Regents meeting in September will reflect changes the
Board developed during its annual retreat. There will be more Full Board time
for policy discussion and presentations by outside experts on significant
issues. There will be fewer
committees. Committee chairs will present the committee reports for Regents
action. The former VESID and EMSC Committees are combined, as are Higher and
Professional Education and Professional Practice Committees.
The Chancellor has appointed new committee leadership and reassigned
committee membership. There are fewer Regents items than in the past, and they
are more concise. These changes complete the cycle of Board development that the
Regents began a year ago when they decided to create a 24-month policy calendar
to guide their work. The Regents Policy Conference precedes the monthly meeting.
BY STATE EDUCATION COMMISSIONER
RICHARD P. MILLS
August Exams
The
August exam period passed without the need for error messages after the fact to
the schools. That’s the way it
should be. Many people contributed
to that result, and many more worked through the summer to create the foundation
for sound exams next January and June. We
aren’t finished, but August shows it can
be done. Our critics are right in demanding
it be done.
When
the August exams were about to go to press, we invited more than two dozen
teachers to review the exams. None had been involved in writing the questions.
All agreed on short notice to put the tests under a microscope. The
teacher reviewers did find some problems with the exams – a diagram not well
labeled, a direction not clear enough, insufficient space for an answer.
These problems wouldn’t invalidate a test, but they were problems
nevertheless. We corrected them and the students never saw them.
The
reviewing teachers weren’t the only contributors to better exams.
State Education Department staff members who aren’t part of the exam
process read every exam before the teachers did – and they added quality. The
exam designers in the State Education Department invented ways to get quick
improvements in quality and they added quality, too. And then there were the printers working two, ten-hour
shifts, seven days a week, who bought enough time for the improvements and still
got the exams in the hands of teachers on exam day.
Improvements
in State Testing
A report to the Regents this month describes actions we have taken and others we will take to improve exam quality, increase SED capacity and provide more information to the public. Here are some of them:
· Quality checks on completed tests. The expert teacher review of the completed tests that worked so well in the August exams will be done from now on for all exams.
·Score validation for the
next Math A. In January we will
collect test results from a sample of student papers immediately after the exam
to check the scoring chart. This will produce immediate statewide information on
how the Math A exam works. An independent panel will review the process. If any
adjustment to the original scale is needed, they will write a public report on
the validity of the change.
· Expanded field testing. We will administer mini-versions of Regents exams close to the actual exam dates, using an approach like one used with 4th and 8th grade exams. This will produce more high quality test questions, and better information about how they worked with the same cohort of students who later take the actual tests.
· Additional staff for testing development. We have reassigned six additional staff to our assessment team. We appointed a seventh person, Howard Goldsmith, as Coordinator of Assessment Operations this month. We are exploring options to obtain the services of more content specialists and plan to add more help in these areas by January.
· Tracking of exam development. A computer system will help identify and resolve delays or bottlenecks that would impede test quality.
· Responding to questions. We will assign people to respond quickly to public information requests and free technical staff to focus on exam development and administration. (This reflects a similar approach that works well in our Office of the Professions.)
In addition, we expect the full
report of the Math A Panel in October will suggest other
improvements and we
will respond quickly to that report, as we did to their interim
report.
Regents Math A
The Regents Math A Panel provided an interim report in August, which addressed two parts of their charge. We immediately used their advice to statistically adjust the scores on the June 2003 Regents Math A to hold students to the same standards applied in June 2002. Before school started, students and their parents and teachers had a table to adjust the scores. This removed the uncertainty for last year’s ninth and tenth graders and enabled schools to schedule them into the appropriate mathematics classes. Now we await the completion of the Panel’s work and their full report, which the Regents will discuss in October. The aim is to find and resolve Math A issues in time for the January and June exam periods. We appreciate the thoughtful work of the Panel.
Physics
– A Policy Issue
A high school physics teacher began a conversation with me and a group of physics teachers this summer with what he called “the threshold question:” For whom is the physics exam intended? Two teachers observed that the Regents physics exam was “a good exam for advanced students,” but to a person, the teachers argued that physics should be an entry-level science. The ensuing discussion raised a policy question that only the Regents can resolve. Here it is: