NOTE: This paper is a re-print of the section in the Technical Appendix to the Regents State Aid Proposal for 2004-05 which described the successful schools study.
Estimating the Additional Cost of Providing an Adequate Education
One of the traditional principles in school finance which has guided Regents Proposal development in past years has been a wealth and need equalization principle. This principle was designed to drive greater amounts of aid per pupil to school districts with limited fiscal capacity and high concentrations of pupils in need. In recent years, however, the focus of school finance, particularly in New York State, has begun to shift from equity to the provision of an adequate education.(23) By the term adequate education is meant the greater equalization of academic outcomes (not resource inputs) so that all children are provided the opportunity to receive an education, which will subsequently allow them to lead meaningful and productive adult lives.
The purpose of this report is to describe the methodology that was used to estimate the likely additional expenditures needed by districts with lower academic performance to achieve educational outcomes that demonstrate that an adequate education is being provided.
Three General Approaches. The literature identifies three basic empirical methods for identifying the cost of providing an adequate education.(24) These methods include:
1) Econometric analyses that use sophisticated statistical techniques to estimate the resource costs associated with different levels of school district performance.
Other strategies are designed to determine the instructional and other costs associated with districts that have already achieved acceptable or adequate performance levels. These approaches are typically of two types:
2) Expenditure per pupil analyses use strategies based upon the gross instructional (and related) expenses of school districts whose achievement meets accepted levels of performance and
3) Professional judgement models employ strategies in which the key instructional components needed to achieve a desired achievement standard are identified by panels of experts, and then costed out. This latter method relies heavily upon the use of professional judgment as to what practices or resources are needed in order to achieve a desired level of academic success and is often referred to as the professional judgement model.
The Econometric Approach: Econometric approaches designed to estimate the cost of achieving a specified academic performance standard are complex, and require the use of two-stage least squares estimation methods. Ultimately, researchers estimate the direct effects or impacts of district characteristics, enrollment characteristics, wealth characteristics, and desired performance requirements on cost per pupil.
Once researchers have estimated these effects statististically, it is possible to insert the actual values of these variables for a given district into a prediction equation – while setting the performance level variable at a desired value – in order to estimate overall cost per pupil. The bottom line is this: when one statistically controls for district-level size and wealth characteristics, the higher the performance expected in the model, the higher the projected costs.
Unfortunately, the results of these more complex
correlational approaches lack transparency, being very difficult to explain to
lay people. As Guthrie and Rothstein have noted, "…when courts demand or
legislatures determine that an adequate education be funded, they will require a
calculation of this adequacy that seems intuitively reasonable, that is
understandable to reasonably well-educated policymakers, and that can be
explained to constituents."(25)
The comments of both Guthrie and Rothstein make clear their view that such
an "ease-of-understanding" standard is not likely to be met by some of these
more complex statistical approaches. In addition, many of the variables
incorporated in these regression models are not particularly intuitive and do
not relate specifically to instructional cost components; consequently, the
results are often viewed as a "black box". That is, while total costs at the
school district level can be estimated by such econometric studies, how these
total costs should be distributed by the state to the district or within the
district to its various school buildings is beyond the scope of such studies.
The Academic Success Approach: Empirical estimates of the cost of an adequate education typically begin by investigating districts that are already achieving a desired state of academic performance. The most straightforward application of the empirical method starts with an examination of the spending patterns among all such districts to determine the average expenditure per pupil of the successfully performing districts. Since districts that perform at high levels often enjoy a very substantial wealth base, and therefore also spend at very high per pupils levels, concerns about technical efficiency are characteristic of this method.
A traditional response to the efficiency concern is to constrain the selection of districts to be analyzed. For example, the districts for which the average expenditure per pupil of successful school districts that would be established could be restricted to the lowest spending 50 percent of such adequately performing districts.
A common variation of this approach is to empirically identify the staffing patterns of academically successful school districts. For example, pupil-teacher ratios, class sizes, number of guidance counselors are some of the patterns that could be examined in a study of this type. Based upon the judgements of SED analysts, normatively appropriate staffing patterns could then be identified and their associated costs calculated. As with the expenditure per pupil approach, it is possible to introduce efficiency into the calculation of cost by limiting the districts analyzed to those who appear to achieve adequate levels of performance at modest cost.
The Professional Judgement Approach: An important variant or extension of the Academic Success Approach relies more heavily on the use of consensus methods and professional judgment to identify the key instructional components to be costed out. Professional judgement methods consist of developing a consensus among professionals as to the appropriate staffing patterns and instructional components needed to achieve academic success. These components are then costed out based upon empirical data in order to estimate overall district-level costs. While this approach benefits politically from significant "buy-in" of the various expert-groups, such a method can be very time-consuming and would require at least one to two years to implement.
Three Critical Methodological Questions
For this study, each of the approaches described above was evaluated. However, in developing an estimate of the expenditures needed to ensure that all districts can provide the opportunity for an adequate education to all students, it was believed that the approach most transparent to the general public would be one based upon demonstrated academic success. The associated expenditures per pupil identified in these successful districts could be modified to reflect regional cost and the educational need of pupils. In short, the study would estimate the expenditures per pupil needed to achieve a specified academic outcome based on the spending patterns of districts actually achieving the specified level of academic performance.
As the methodology was developed, researchers answered three questions involving very specific operational definitions of major concepts. The questions were:
How should academic performance be measured?
How should pupil need be addressed? and,
Should there be a regional cost adjustment?
Measurement of Academic Performance
A critical methodological issue addressed by the study concerned the measurement of academic performance. New York State is presently implementing a series of tests designed to measure academic performance at various grade levels. Examples of such examinations include:
·English Language Arts and Mathematics (fourth grade)
Fourth Grade Tests. Fourth grade test results can be grouped into four categories or performance levels. These performance categories are:
·Level 1---Does not meet the standards;
High School Regents Examinations. Several important issues had to be addressed in using the results of high school examinations as components in the operational definition of an adequate education. First, results on Regents exams are given as a numerical score only. Scores are not automatically translated into levels of performance. Based on a review of the School District Report Card and the Annual Report to the Governor and Legislature on the Educational Status of the State’s Schools the classification system shown below for high school Regents exams was developed by this study. The researchers concluded that these classifications best approximated the four-level scoring system that exists for elementary and middle school students.
The classifications are:
·Level 1 = a score of 0 to 54
Data on Regents High School examinations were collected for five tests. The tests were:
A potential problem with using single-year test results, of course, is that academic outcomes in any one-year may be atypical and more reflective of a one-time phenomena rather than a typical example of academic outcomes over a multi-year period. This traditional critique was addressed for this study by using a three-year average of test results. Test results used in the study were from the 1999-00, 2000-01 and 2001-02 school years.
Ultimately, to make a cost estimate, adequate education needs to be defined in quantitative terms. In establishing its definition, the study had two basic choices. It could use either test scores or the percent of test takers achieving a specified educational result. Use of either measure would be valid. However, since the Court of Appeals in the Campaign for Fiscal Equity court ruling indicated that every child should be provided with an adequate education, it would appear that a threshold measure which captures the percent of test takers achieving a specified standard would be the most appropriate measure to use.
Upon reaching this decision, the study addressed three questions:
1. What level of achievement should be reached?
2. What percent of students should attain the specified outcome? And,
3. What tests should be used?
If students in a district are receiving an adequate education, it would seem that the vast majority of its students should be capable of achieving the Regents standards. This means, on whatever tests one uses for defining academic outcomes, the vast preponderance of students should be scoring at the equivalent of level 3 or level 4. So for this study, it was believed that if a district had on average 80 percent of its students scoring at level 3 or higher on the specified tests, the district would be considered as providing an adequate education.
Finally, the study had to determine which specific examinations would be used in developing the cost estimate. It was decided:
To use both fourth grade tests in the definition of an adequate education. This decision was made primarily because only the central high districts do not have a fourth grade. Only one district was lacking fourth grade data. Thus almost every district would have fourth grade data, which would be a strong indicator of whether students had or had not acquired a sufficiently strong educational foundation to insure that high school graduation requirements were likely to be met; and,
To use the test results of the five high school examinations previously listed, since passing of these or similar tests is required for high school graduation.
Missing Data. An important issue from a methodological perspective was how to treat a district if it were missing data. Missing data could occur because of several factors. These factors include:
Grade configuration of a district. A K-6 district would not have eighth grade or high school results. Conversely, a central high school district would not have any fourth grade results. In a sense, the district wasn’t missing data as much as the data were non-existent for the district. Grade configuration was a major factor in missing data. For example, of the five districts without any data for either the fourth grade tests, four were central high schools.
Data were truly missing. No test data exists for one district. Other data may be missing due to administrative error or because a particular test was not given in a district for one or more years.
Based on these circumstances, the following decisions were made:
If absolutely no test data existed for a district on any of the tests used, it would not be included in the study. Kiryas Joel was the only district not included in the study for this reason.
If a district had some test data, the determination concerning provision of an adequate education would be based on existing data.
Operational Definition of an Adequate Education
Based on all of the considerations described above, an adequate education was operationally defined as a district:
With a simple, unweighted average of 80 percent of its test takers scoring at Level 3 or above on seven examinations (Fourth Grade English Language Arts, Fourth Grade Mathematics, high school Mathematics A, Global History, U.S. History, English and Earth Science) in 1999-00, 2000-01 and 2001-02. The reader will note that, given this operational definition, a district could have less than 80 percent of its test takers with a score below Level 3 on one or more of the individual tests and could still be found as providing an adequate education.
Although this definition does not meet the Regents goal that all students achieve the standards, it does identify districts where the opportunity to achieve exists. Thus this operational definition can be viewed as a reasonable compromise.
If student need is believed to be an important issue in understanding academic performance two methodological questions concerning the quantification of need must be addressed. The questions are:
·What type(s) of students best reflect student need?
What Pupil Count Should be Used to Measure Need? An assortment of measures could be used to estimate student need. Each of the possible counts possess strengths and weaknesses. A common measure used to identify student need among the 50 states is the percent of students eligible for a free and reduced price lunch. Indeed, in New York State, the K-6 percent of students eligible for a free or reduced price lunch is one of the pupil counts used to allocate a supplement to Operating Aid to help districts meet the needs of at risk students, known as Extraordinary Needs Aid. For these reasons, the study concluded student need could best be measured by the percent of K-6 pupils eligible for a free and reduced price lunch.
The count of K-6 students eligible for a free or reduced price lunch, however, may be subject to wide variation in some districts. For this reason, average counts reflecting three school years were used. Such an average would minimize the possibility of grossly misidentifying a district’s poverty rate due to a unique circumstance. K-12 districts that did not provide a school lunch program in 1999-00, 2000-01 and 2001-02 were given a K-6 free and reduced percent of zero. Central high school districts were given the average count of their component school districts.
What Should Be the Additional Weighting for Need? To incorporate "need" into a student count requires the development of an additional weighting. In school finance, the term additional weighting is usually associated with the quantification of the extra costs associated with providing a specified service. These extra costs are then translated into an additional weighting. For example, secondary students (grades 7-12) in New York State are provided an additional weighting of 0.25. This means a secondary pupil in certain student counts used in state aid formulas has a calculated value of 1.25 (1.0 + 0.25).
The additional weighting selected is critical in determining the cost of an adequate education. This immediately raises the question of what is the appropriate additional weighting for need. In seeking guidance for a suitable need weighting, we have two sources - existing practice and the research literature.
The legislation of other states concerning the additional weighting of poverty or at-risk pupils is another source to consider in determining the appropriate additional weighting for such students. Carey described the practices of states as of the 2001-02 school year4 and found that the funding level for poverty-based education aid varied widely among the states. In his view this was often more a reflection of available resources than of the actual costs of educating such students.
Since the 2001-02 school year, several states have taken legislative action concerning poverty or at-risk pupils. Maximum additional weightings enacted for poverty or at-risk pupils have ranged from 0.25 to 1.0. In New Hampshire and Wyoming the concept of a variable additional weighting for need based on the concentration of poverty pupils has been introduced.
Although a wide range exists in the research literature in terms of the appropriate additional weighting for student need, most of the literature suggests an additional weighting of at least 1.0. Indeed, in September 2003 the State Education Department released a study on educational need, expenditures per pupil and educational achievement in which student need was given an additional weighting of 1.0.
For these reasons it was decided that pupils would be given an additional weighting of 1.0 for poverty (based on 1999-00, 2000-01 and 2001-02 K-6 students eligible for free and reduced price lunch).
In recent years, the Board of Regents in its State Aid proposal has also endorsed the concept of adjusting State Aid to reflect the variation in regional cost found to exist in New York State. It has done so due to the dramatically different costs associated with educating students in various geographic regions of the State.
To properly reflect these differing educational costs, it was decided to incorporate regional cost into the cost estimates. The cost indices used in calculating the estimate are the Regional Cost Indices (RCI)7 calculated for the 2004-05 State Aid Proposal of the Board of Regents. The RCIs were calculated based upon labor force regions as these have been defined by the New York State Department of Labor. The RCIs calculated for these labor force regions have been normed to a "North Country standard" and are described in Table 1 below:
Table 1: Regional Cost Indices for Labor Force regions in New York State:
North Country 1.000
Mohawk Valley 1.016
Southern Tier 1.061
Western NY 1.080
Central NY 1.132
Capital District 1.168
Finger Lakes 1.181
Hudson Valley 1.359
Long Island/New York City 1.496
Expenditures Per Need-Adjusted Pupil
The final approach was to develop an "expenditure per need adjusted pupil" model, which compared the expenditure pattern of districts with acceptable academic performance to districts with educational performance below the stated standard. Expenditures were defined as general education instructional expenditures8 (including an estimated amount for fringe benefits) as adjusted by the Regents Regional Cost Index calculated in 2003. The pupil count was the same count used for general education instruction as defined in statute for the Fiscal Supplement to the School Report Card.9 This count was then adjusted to reflect student need by weighting the free and reduced price lunch count at 1.0.
A graph of this prototype is shown in Figure 1. Under this approach, the first step was to identify districts providing an adequate education. As noted earlier, such districts were defined as districts in which an average of 80 percent of the students taking the seven previously identified examinations had a score that was at Level 3 or above. Districts in which on average 80 percent of the students tested did not score at levels 3 or 4 were identified as districts which may need to increase instructional expenditures in order to improve academic performance.
The next step in the methodology was to calculate the mean need and cost-adjusted instructional expenditure per pupil for all districts classified as providing an adequate education. These districts were then ranked from high to low on need and cost-adjusted instructional expenditures per pupil. At this point an efficiency measure was introduced. The mean expenditure per pupil was calculated for the lower half of these districts, based on per-pupil expenditures.
Thus, the procedures followed by the study to estimate the amount of additional instructional expenditures required to achieve adequacy can be figuratively expressed as shown in Figure 1.
Figure 1: Estimating the Increase in Instructional
Needed So That the Opportunity for an adequate Education
is Provided by All Districts
|Identify High Performing Districts||
|Identification of Other Districts|
|Convert Expenditures into Cost Adjusted $|
|Adjust Pupil Count to Reflect Need|
Expenditure/Pupil Patterns of High
Performing Districts 9; 9; 9; 9; 9; 9;
|Apply Any Efficiency Criteria|
|"Spending Per Pupil Gap" Analysis|
Determine Dollar Increase
Needed in Cost and Need
(Per Pupil Need x Need
Needed into Actual Dollars
(Cost and Need Adjusted Dollars
return to FARU homepage