Table of Contents

Introduction..................................................................................................................................................... 1

Background.................................................................................................................................................... 1

Objective, Scope and Methodology.................................................................................................. 1

Comments of Orange-Ulster BOCES Officials............................................................................. 2

Report Card........................................................................................................................................................ 3

Data Accuracy and Supporting Documentation...................................................................... 3

Career and Occupational Education Section............................................................................... 3

Alternative Education Section............................................................................................................. 4

Adult Education Section.......................................................................................................................... 5

Recommendations....................................................................................................................................... 6

Comments of Orange-Ulster BOCES Officials............................................................................. 6

Chapter 602 Report........................................................................................................................................ 7

Occupational Education Section....................................................................................................... 7

Special Education Section...................................................................................................................... 7

Itinerant Services Section – Visually Impaired Teacher...................................................... 7

General Education Section – Alternative Education........................................................... 7

Recommendation.......................................................................................................................................... 9

Comments of Orange-Ulster BOCES Officials............................................................................. 9

Report Card and Chapter 602 Report............................................................................................. 10

Discrepancies Between the Report Card and the Chapter 602 Report...................... 10

Recommendation....................................................................................................................................... 11

Comments of Orange-Ulster BOCES Officials........................................................................... 11

Employment Preparation Education (EPE)................................................................................ 12

Reported Contact Hours Were Overstated............................................................................... 12

Computer Database Does Not Support EPE Contact Hours............................................... 13

Contact Hour Documentation........................................................................................................... 14

Class Size......................................................................................................................................................... 15

Recommendations..................................................................................................................................... 15

Comments of Orange-Ulster BOCES Officials........................................................................... 15

Opportunities to Improve Management Processes............................................................ 16

Inappropriate Accounting and Reporting of Related Service Costs......................... 16

Use of Unapproved Criteria/Guide Numbers for Special Class – Option III.............. 16

No Written Policies and Procedures in Accounting and Personnel Areas........... 17

Lack of Implementation of Some Independent Auditor Recommendations.......... 17

Recommendations..................................................................................................................................... 18

Comments of Orange-Ulster BOCES Officials........................................................................... 19

Data Collection and Use......................................................................................................................... 20

Follow Up - Previous Audit Findings.............................................................................................. 24

Recommendation Implementation Plan...................................................................................... 24

Appendix A – Contributors to the Report

Appendix B – Comments of Orange-Ulster BOCES Officials

Appendix C – Audit Review Proceedings

Introduction

Background

Public and legislative attention has been turned to data provided by Boards of Cooperative Educational Services (BOCES) and school districts with the issuance of Report Cards. Chapter 436 of the Laws of 1997 made BOCES Report Cards available to the public on April 16, 1999. Graphically presented, the Report Cards summarize program accomplishments and services BOCES provide for children and adults.

In addition, Chapter 602 of the Laws of 1994 added Section 215-b to the Education Law to require the Commissioner of Education to prepare and submit to the Governor, the President Pro Tem of the Senate, and the Speaker of the Assembly, an annual report beginning January 1, 1996. The Chapter 602 Report details certain financial and statistical outcomes of BOCES, tuition costs for selected programs, per pupil cost information, and aggregate expenditure data for BOCES’ administrative, capital, and service functions. The Chapter 602 Report also includes changes from the year previous to the report year for all data.

BOCES are organized under Section 1950 of the Education Law. Chapter 474 of the Laws of 1996 amended Section 305 of the Education Law to require the New York State Education Department (Department) to perform fiscal audits of BOCES at least once every three years.

Orange-Ulster BOCES is headquartered in Goshen, New York and serves 18 component districts which have enrollments of more than 50,000 students. Orange-Ulster BOCES ranked 11th largest in the State, in terms of total general fund expenditures, for the 1997-98 school year.

Objective, Scope and Methodology

Pursuant to Sections 305 and 1950 of the Education Law, we audited management practices, records and documentation related to the data contained in the BOCES Report Card and the Chapter 602 Report for the period July 1, 1997 through June 30, 1998, and we followed up on the status of previous recommendations made in audit report number BOC–0496-2. We also reviewed the Employment Preparation Education (EPE) Program for the period July 1, 1997 through June 30, 1998. The objectives were to:

· evaluate the adequacy of management controls over data reported to the Department and obtain reasonable assurance that the data are valid and reliable.

· determine how BOCES use data to monitor the performance of the schools within its supervisory district.

· determine how BOCES use data to monitor student outcomes.

· evaluate the wide range of costs identified in the Chapter 602 Report and understand the reasons for the variances.

· determine if EPE claims are adequately documented.

· follow up on the findings and recommendations from the previous audit to determine whether timely and adequate corrective actions were implemented.

To accomplish our objectives, we reviewed applicable laws, regulations, policies and procedures; interviewed Department and BOCES’ management and staff; examined records and supporting documentation; sampled transactions on a non-statistical basis; and reviewed Orange-Ulster BOCES audited financial statements.

We conducted the audit in accordance with Government Auditing Standards issued by the Comptroller General of the United States. An audit includes examining, on a test basis, evidence supporting records and applying other audit procedures considered necessary in the circumstances. An audit also includes assessing the estimates, judgments and decisions made by management. We believe that the audit provides a reasonable basis for our findings, conclusions, and recommendations.

Comments of Orange-Ulster BOCES Officials

Orange-Ulster BOCES officials generally concur with the findings of this report. Their comments on the matters contained in this report have been included where appropriate. Their final comments have been included as Appendix B to this report.

Report Card

The BOCES Report Card is intended to be used as a tool for the continuous improvement of BOCES programs and services, and for providing information to communities. The Report Card displays the following programs and services: Career/Occupational Education, Alternative Education, Adult Education, Special Education, Curriculum and Professional Development, Technology, the State Testing Program, and the 1996-97 Expended Budget. The State Testing figures are provided by the Department, not Orange-Ulster BOCES, so those figures were not included in the scope of the audit.

An objective of this audit was to determine the accuracy of the Report Card data and the adequacy of the supporting documentation. The audit found the following areas that need improvement in the Report Card.

Data Accuracy and Supporting Documentation

Data included in the Report Card must be valid and reliable in order to be used effectively. BOCES officials are given the opportunity to review the Report Card prior to its release. Each section displays charts and graphs to present data on the enrollment, completion, tuition, and/or data expressed as a percentage of local totals or statewide totals.

Career and Occupational Education Section

Career/Occupational Education is a two-year program for secondary students in grades 11 and 12. The Department did not specify which date to use for enrollment figures or which students to include or exclude in the data reported. Tuition data are taken from the Chapter 602 Report.

· Orange-Ulster BOCES’ enrollment and completers reported under the Completion Rates section are not accurate. The Report Card shows reported enrollment as 5,748 and completers as 2,115 when data provided during the audit showed 1,291 for enrollment and 455 for completers.

The audit determined that Orange-Ulster BOCES incorrectly combined the enrollment and completers for all programs (Career/Occupational Education, Adult Education, and Special Education).

· Orange-Ulster BOCES provided a list of 453 Career/Occupational Education completers which is 2 less than the 455 completers on the form originally provided to the audit. However, the Department’s report “Career Ed-2 (Secondary) – Follow up Report for Secondary Students Who Graduate in 1998” submitted by Orange-Ulster BOCES only shows 433 completers. This is 20 fewer than the supporting list of student names and 22 fewer than the data supplied to the audit.

The Report Card, with the incorrect enrollment and completers information, was submitted to Orange-Ulster BOCES before its release but officials failed to notify Information and Reporting Technology Services (IRTS) that the numbers were incorrect before the final release of the Report Card. When the information presented in the Report Card is not valid or reliable, the Report Card cannot effectively be used as a tool for the continuous improvement of BOCES programs and services, or used to provide reliable information to communities.

Orange-Ulster BOCES officials have changed their procedures so the Directors of Career Education and Special Education work together in order to ensure that numbers are correct and not duplicated. They stated they will also use the same date for all enrollment data. In addition, they will define the various categories of students and will use consistent definitions for student categories such as completers, dropouts, and graduates.

Alternative Education Section

BOCES operate alternative school programs for students at risk of dropping out. The objective is to retain these students in the Alternative Education Program until they graduate or return to a regular school setting. The Department did not specify a date for enrollment figures. Enrollment and tuition are shown for three years.

· Orange-Ulster BOCES did not report any students receiving a high school diploma in the “Student Outcomes” section for the 1997-98 school year, but reported the number of Alternative Education students who received diplomas during the 1995-96 school year. The audit was informed that this information was not reported because Orange-Ulster BOCES does not actually award high school diplomas. They only recommend to the student’s component district that the student receive a diploma.

Orange-Ulster BOCES should obtain this information so the students’ records can be updated to reflect the fact that the students received a high school diploma. In addition, the total number of students that are reported in the “Student Outcomes” section should be equal to the enrollment. However, without reporting on the students who received high school diplomas, they cannot be equal.

· Orange-Ulster BOCES was able to provide a list of names that supports the enrollment of 159 students for Alternative Education with only an immaterial exception (one student was included twice). However, the breakdown on this list did not support the breakdown of students in the “Student Outcomes” section. See the chart below for comparison.

Per 97-98 Report Card		Diff	Per BOCES List of Names
HS Diploma	0	(31)	31		Graduating Seniors
GED	31			33	Students that received a GED Diploma (includes using a student’s name twice)
				(1)	Less duplicate
		(1)	32
Exited	21	21
Remained in BOCES Program or returned to school district	74	(21)		70	Students that remained in BOCES Program
				5	Transportation and Learning Center Projected Enrollment
				20	Commerce Learning Center Projected Enrollment
			95
Total	126	(32)	158		Total
Alternative Ed Student Enrollment	159	1	158		Total

Orange-Ulster BOCES agrees that the Alternative Education data were incorrect and states they will report information consistent with Department definitions and guidelines. They will also track students and review the data before submission to make sure that outcomes information agrees with the total number of students.

Adult Education Section

BOCES sponsor both Adult Occupational Education programs and Adult Basic Education courses. Placement results for the 1997-98 Adult Occupational Education programs will not be available until late Spring because agencies have 180 days from the date of graduation to follow-up on any program that requires a license or exam. The results were reported in the 1998-99 Report Card.

· The unduplicated Adult Education enrollment was shown as 2,556 when it should have been 3,374. The incorrect figure was the result of a deduction made by the Department for 818 students labeled as “Unknown” for ethnic group and gender. The Report Card, with the incorrect enrollment and completers information, was submitted to Orange-Ulster BOCES before its release but officials failed to notify IRTS that the numbers were incorrect before the final release of the Report Card.

· Under the Improved Literacy Results section, the Improved Literacy column for Reading Level 6-8.9 was reported as 295 but should have been 452.

The information for the Improved Literacy Results table comes directly from the “Adult Basic and Adult Secondary Program Report.” When IRTS is preparing the table, it requests the data from Office of Workforce Preparation and Continuing Education (OWPCE). In this case, OWPCE provided the incorrect information for the Improved Literacy column for Reading Level 6-8.9. However, Orange-Ulster BOCES did not notice the error so it could be corrected before the Report Card was released.

· The Report Card showed that 74 students obtained part-time employment and 61 students obtained full-time employment (a total of 135 students). However, the audit reviewed the list of 135 names and found that only 24 students obtained part-time employment (50 less than the Report Card), and 111 obtained full-time employment (50 more than the Report Card).

· Orange-Ulster BOCES supplied a computer database containing information related to the Adult Education programs. The audit found that, with minor exceptions, the database had more students than were reported on the “Adult Basic and Adult Secondary Program Report” for each area reviewed. In addition, 63 students on the computer database had both male and female designations assigned to them and/or were assigned two different ethnic groups or an ethnic group and “Unknown.” The audit could not determine if this resulted in duplicate counts on the “Adult Basic and Adult Secondary Program Report.”

Orange-Ulster BOCES agrees the Adult Education data are incorrect. They state the problem was faulty computer software used to track these data. They have purchased a new software program and have trained personnel in how to use the software. They have also instituted new administrative procedures for enrolling and registering Adult Education students that allow them to track such items as gender, ethnicity, reading level, employment status, veteran status, disability status, as well as several other categories.

Recommendations

1. Review data on the draft Report Card to ensure accuracy.

2. Report the number of students receiving a high school diploma for Alternative Education.

3. Maintain accurate data for Adult Education.

Comments of Orange-Ulster BOCES Officials

Orange-Ulster BOCES officials concur with these recommendations.

Chapter 602 Report

The Chapter 602 Report is compiled by the Department’s BOCES and School District Organization (BSDO) unit. The information used by BSDO comes primarily from the SA-111 and “Data Collection Forms” submitted by the BOCES.

The report details certain financial and statistical outcomes of BOCES, tuition costs for selected programs, per pupil cost information, and aggregate expenditure data for BOCES administrative, capital and service functions. In accordance with the statute, the Chapter 602 Report is to include changes from the previous year to the report year for all data.

An objective of this audit was to evaluate wide ranges of costs identified in the Chapter 602 Report and understand reasons for the variances. Another objective was to determine the accuracy of the Chapter 602 Report data and the adequacy of the supporting documentation. The following describe the audit’s findings.

Occupational Education Section

· Orange-Ulster BOCES did not maintain documentation to support the enrollment used to calculate the tuition rates for the Occupational Education program.

Special Education Section

· Orange-Ulster BOCES did not maintain documentation to support the enrollment used to calculate the tuition rates for Special Class Options II, III, and IV.

· Orange-Ulster BOCES did not maintain documentation to support the number of sessions used when calculating the tuition rate for Related Services (Speech Pathology, Physical Therapy, Occupational Therapy, and Counseling Services (including Rehabilitation)).

Itinerant Services Section – Visually Impaired Teacher

· Orange-Ulster BOCES did not maintain supporting documentation for or the methodology used to calculate the tuition rate for the Visually Impaired Teacher.

The Department’s BSDO requested tuition rate information on activity code 4650 for the Speech Impaired Teacher. However, Orange-Ulster BOCES provided tuition rate information on activity code 5130. Therefore, the tuition rate that is reported on the Chapter 602 Report for Speech Impaired Teacher is not correct since it is based on the incorrect activity code.

General Education Section – Alternative Education

· Orange-Ulster BOCES did not maintain supporting documentation for the information used to calculate the tuition rate for Alternative Education.

The Department’s BSDO requested tuition rate information on Criteria/Guide Numbers 5873 and 5878 for Alternative Education. However, Orange-Ulster BOCES provided tuition rate information on Criteria/Guide Number 5916.

The audit requested further clarification from Orange-Ulster BOCES regarding the services provided under Criteria/Guide Number 5916 and the type of students receiving these services. However, Orange-Ulster BOCES did not supply this information and the audit was not able to determine if Orange-Ulster BOCES was using the incorrect Criteria/Guide Number for the services or the incorrect Criteria/Guide Number when reporting the tuition rate for Alternative Education.

The audit could not verify if the tuition rates for the above areas were based on the appropriate attendance and cost data. As a result of supporting documentation not being maintained, the above-mentioned Chapter 602 Report tuition rates and other reported data cannot be relied upon to make meaningful comparisons with other BOCES.

Orange-Ulster BOCES officials state they followed the directions given by the Department for reporting Chapter 602 Report information. They said they were unaware they had to show how they obtained the tuition rates and had to maintain documentation. They state they will attach copies of supporting documentation to requested forms in the future.

The supporting documentation is to be maintained by the BOCES to be provided upon request by the Department or an audit.

Recommendation

4. Maintain documentation to support tuition rates reported to the Department.

Comments of Orange-Ulster BOCES Officials

Orange-Ulster BOCES officials concur with this recommendation.

Report Card and Chapter 602 Report

IRTS uses information from the Chapter 602 Report when preparing the Report Card. Consequently, the underlying nature of the information that is submitted to the Department for the Chapter 602 Report should be comparable to the underlying nature of the information that is submitted to the Department for the Report Card. If it is not, information reported in the Report Card could be misleading and cause users to come to incorrect or unreasonable conclusions about the performance of the BOCES.

Discrepancies Between the Report Card and the Chapter 602 Report

Orange-Ulster BOCES claims the pupil count for the Chapter 602 Report is based on billings and Full Time Equivalent students whereas the enrollment for the Report Card is taken at a specific date. As a result, the following discrepancies were noted between enrollment on the Chapter 602 Report and the Report Card.

Section	Year	Report Card (enrollment)	Chapter 602 Report (pupil count)	Difference
Career/ Occupational Education	96-97	1,264 (Based on OEDS report)	1,172	92
Career/ Occupational Education	97-98	1,292 (Based on OEDS report)	1,254	38
Alternative Education	97-98	159 (Based on submission of BOCES)	115	44

Because the tuition rate on the Report Card comes from the Chapter 602 Report and the enrollments are somewhat different, the tuition rate on the Report Card is not a true reflection of the enrollment on the Report Card. Therefore, the tuition rate should not be used for accurate analysis, such as projecting the costs of the program or when comparing tuition rates among BOCES. Orange-Ulster BOCES needs to ensure it is using the same method to measure attendance for both the Report Card and the Chapter 602 Report.

Orange-Ulster BOCES reported “NA” (meaning not available) for pupils and tuition per pupil under Alternative Education for the 1996-97 school year on the data collection forms BSDO uses to prepare the Chapter 602 Report. Enrollment and tuition were reported for Alternative Education for the 1996-97 school year on the Report Card. However, because of the difference in information reported on the Report Card and the Chapter 602 Report, the information could be incorrectly interpreted.

Orange-Ulster BOCES officials agree there were discrepancies between the Chapter 602 Report and the Report Card. They believe the reason for the discrepancies is that the Chapter 602 Report is a net tuition (based on actual costs after refunds) and actual participation report. The Report Card, on the other hand, uses budgeted tuition rates based on an estimated number of students and/or an actual number of students at a specific point in time. They will check to ensure that the Report Card information is consistent with the Chapter 602 Report information.