The vendor shall provide a Test Examiner’s Manuals for the tests. Manuals will contain instructions for an administration for the appropriate exams. They also will include information about security of materials. The manuals will be available in an electronic format accessible via the Agency’s website prior to the hardcopy distribution. These manuals will also be provided to each County Test Coordinator. The vendor shall ensure manual proofs are free of typographical and format errors before they are submitted to the Agency. An example of Test Examiner’s Manuals must be provided for committee review.
22.214.171.124.4 Interpretative Guide
Interpretative guides for the grades 8 and 10 college predictive tests and the college entrance exam will be provided to assist county and school personnel in reading and using program reports. The vendor is responsible for distributing this report to each school (state, public or private) participating in the assessments. An example of Interpretative Guide must be provided for committee review.
126.96.36.199.5 Other Ancillary Documents
The vendor will also provide a listing of all ancillary documents they currently provide to county school districts for the proposed assessments. The vendor will print and disseminate any additional materials needed to implement the project, such as directions to the student to complete the answer documents, transmittal memoranda, labels for packing and packing lists. These documents are to include, not limited to:
District Header Sheet
School Header Sheet
188.8.131.52.6 Braille and Large Print Documents
For student level test documents, the vendor will provide sufficient quantities of Braille, as outlined in Sections 184.108.40.206.8.1 and in 220.127.116.11.8.2 and large print versions in size 18 font at each grade level. Test administrator notes and scripts accompanying the Braille test versions will also be provided by the vendor.
The development contractor will provide Braille and large print versions of various documents referred to elsewhere in this RFP, in addition to the test forms as needed.
18.104.22.168.7 Breach Form
No breach forms have been requested for grades 8 and 10 predictive tests nor for the grade 11 college entrance exam. Students in grades 8 and 10 who have invalidated test scores will not be re-assessed with a breach form.
22.214.171.124 Materials Distribution/Retrieval
Many forms and materials are needed to implement the program. Some of the materials listed in this section will assist schools, counties and the state in implementing quality control procedures and will ensure the integrity of the data collected by the program. The Agency also uses special forms to evaluate the quality of the assessment program and its implementation annually.
The vendor must provide a plan for materials distribution and retrieval. The vendor is responsible for all arrangements and costs associated with packing, distributing, and returning materials. Prompt and accurate delivery of materials is important to Agency and to local county/school personnel who have the responsibility of managing test materials. There must be 100 percent accounting of all test booklets and answer documents returned by the counties.
The vendor must describe the process and the accountability system used to monitor the return of documents by the counties. The vendor must guarantee distribution procedures are acceptable. Vendors’ proposals must include descriptions of the procedures they will use to complete these tasks.
The vendor will provide
technical services related to the production and interpretation of results,
plan to integrate WVEIS student information with the scoring system and exporting student score files back into the WVEIS files,
use of WVEIS unique student identifier, and
log in process for college entrance exam assessment materials.
126.96.36.199 Pre-identification of Answer Documents
The vendor will be responsible for pre-identifying answer documents using data transmitted via an electronic medium from the Agency for each administration. The vendor will assume pre-identification will be used for 100 percent of the public school population. Answer documents for students that have transferred, are home-schooled/home instructed, or attend a private or parochial school will be manually hand-gridded.
The Agency will submit estimated counts for print materials for the grades 8 and 10 college predictive tests in July. A file will be submitted in October for labels and pre-identified answer documents. The vendor will then send answer documents or labels (packaged by school in alphabetical order by student last name) to arrive in counties two weeks before the test administration.
The Agency will submit pre-identification files approximately five months prior to testing to provide estimated counts for print materials for the grade 11 college entrance exam. A second file will be submitted approximately two months prior to testing for labels and pre-identified answer documents. The vendor will then send answer documents or labels (packaged by school in alphabetical order by student last name) to arrive in counties two weeks before the test administration.
The vendor will establish a system to ensure the pre-identified answer documents/labels delivered to counties contain accurate data, reflect the options selected by counties, are accurate, and are printed at a level of quality that permits accurate scanning and precludes the possibility of smudging. To help ensure all data fields include acceptable data, the vendor will provide the Agency with a checking program to be used prior to submitting data to the vendor. The vendor must establish a system to allow the Agency to transmit pre-identification data electronically over a data transmission network that will be secure and accessible only to authorized users. The vendor will bear the cost of establishing the system and providing network-specific software needed by the Agency to access the system.
Vendor will provide evidence of quality control procedures throughout the printing process to ensure the quality of label printing.
188.8.131.52 Optional Technology Systems
Vendors shall also propose solutions and other technologies to permit the pre-identified answer documents/labels to be printed on dates closer to the actual test administration dates. Such approaches might utilize the Internet or FTP sites to upload data to the vendor or provide mechanisms whereby the labels can be printed at the state level, with appropriate quality controls. Proposals for other technologies to pre-identify answer documents are acceptable as long as the alternative provides at least the same level of timeliness, reliability.
184.108.40.206.1 FTP Site
Vendors must include provisions for a secure File Transfer Protocol site for data, documentation and report transfer from the vendor to the Agency and from Agency to the vendor. This site must be maintained by the vendor to ensure its continued availability throughout the life of the program.
220.127.116.11 Pack and Distribute Materials
18.104.22.168.1 Packaging Specifications
The vendor will prepare packaging specifications to be delivered to the Agency in July before each operational administration of the grades 8 and 10 college predictive tests. For the college entrance exam for grade 11, the packing specifications will be delivered to the Agency no later than three months prior to testing.
The packaging specifications will be updated as required to meet vendor and Agency needs. The packaging specifications will include the vendor’s procedures for packing and distributing materials to counties and receiving the materials from the counties. The specifications will include a description of how the materials are packed, examples of packing and inventory lists for boxes to counties and schools, methods used for distributing and returning materials, and a description of the procedures used to inventory materials as they are returned.
The Agency will provide the vendor with a list of the current names, mailing and shipping addresses, email addresses, and phone and fax numbers of the County Test Coordinators. These persons monitor all aspects of the assessment for their counties, the Institutional Schools, and the Schools for the Deaf and Blind. Vendors must ship materials to county test coordinators at approximately 57 separate sites or to the high schools. The number of schools may change slightly during the life of the contract.
All materials for use by private and parochial students or schools will be shipped to the county test coordinator of the appropriate county or to the high school.
Table 40 provides information about the estimated number of students to be the recipients of testing materials.
Table 40: Number of Public, Private, and Parochial Students Based on 2006-2007 Enrollment Figures for Grades 8, 10, 11 and 12
Public, Private, and Parochial Students plus 5% overage
To be shipped to county test coordinator
To be shipped to county test coordinator
To be shipped to school
To be shipped to school
*It is expected that the 12thgrade make-up will be for students who as 11th graders were absent on the West Virginia test date and the retest will be only for students who did not receive the acceptable college benchmark on the West Virginia grade 11 college entrance exam.
The vendor will submit to the Agency the number of requested testing materials by school, county, and grade at least two weeks prior to the administration of the tests.
22.214.171.124.3 List of County Test Coordinators
The vendor will be responsible for maintaining a list of County Test Coordinators (CTCs) and updating it as notified by the Agency. The most current directory of the CTCs will be sent to the vendor each year by August. At the beginning of the contract the Agency will provide the vendor with a data file containing a list of the counties and schools (names and identification numbers) and the numbers of students by grade tested during the last year. Currently, all state wide test materials for the county are sent to the county test coordinator; the vendor will propose their delivery system and current state delivery system of sending all materials through county test coordinators.
126.96.36.199.4 Packing, Distributing, and Receiving Materials Provisions
The vendor will pay charges on all materials shipped to and from the County Test Coordinators and the Agency for the grades 8 and 10 college predictive assessments. Materials will be delivered either to the CTCs or to the Agency. The vendor will pay charges on all materials shipped to county or the school testing sites or CTCs for the grade 11 college entrance exam and the grade 12 make-up/retest college entrance exam.
The vendor will make arrangements for and pay for shipment if, due to a delivery error, the county is asked to ship materials to another county. The vendor must use an overnight delivery service for such shipments. Vendors will identify carriers they propose to use and the procedures for delivery and return of testing materials. The Agency must approve all carriers.
The vendor is not responsible for any costs schools may incur in shipping test materials from their schools to the county office, unless the need to ship is the result of a packaging error by the vendor. The vendor will package materials for each shipment by school in returnable boxes and/or supply mailing envelopes.
The vendor will provide an online system and toll-free phone line for county staff to order additional materials. The WVEIS file will be used to generate the quantity of testing materials per district. If the WVEIS files have not been updated and the Agency does not have the correct order information, the school/county may order extra material at their expense if they gave Agency incorrect data. At the conclusion of the test administrations, the vendor will prepare and submit to the Agency a reporting of the requests by county for additional materials and the county costs.
The vendor will staff toll-free phone and fax lines during the period in which materials are shipped, additional materials are ordered, and materials are picked up from schools and counties. The service will utilize one to three individuals, as needed, who are designated to respond only to West Virginia testing calls and perform other West Virginia work during this period. The vendor’s telephone lines must be staffed during Eastern Time Zone working hours.
The vendor will prepay charges on return shipments from the counties. Return labels, prepaid postage labels, freight bills-of-lading or web-based systems and instructions will be provided for the materials being returned.
The vendor will be responsible for mailing or shipping by overnight delivery service or other means, as appropriate, any miscellaneous materials to the Agency and counties as situations arise. The vendor must secure the services of shippers who will provide inside delivery and unload large shipments onto loading docks. The vendor must develop procedures to monitor the receipt of all materials and develop error logs. The date materials are received, who signed for the delivery and any errors made by counties in packaging and completing forms must be documented. When problems arise, the vendor will be responsible for contacting the counties and the Agency concerning the problem and resolving the problem. The error logs will identify by school and counties all failures to follow the established procedures and, if appropriate, how the errors were resolved. The error logs will be delivered to the Agency immediately after materials from all counties have been received.
The vendor will verify, by number, all test books and/or secure answer documents returned after testing, using an acceptable system, such as bar codes. The vendor will provide a written report to the Agency documenting the check-in of all secure materials within 60 calendar days of initial receipt of the materials. If the vendor’s system is demonstrated to be unreliable, at the vendor’s expense, each document will be required to be scanned twice, creating independent data files that can be matched for scanning errors. Beyond these measures, the vendor will be required to manually check answer documents to ensure 100% accuracy of check-in for secure documents. Upon Agency request, the vendor will work to resolve discrepancies in the numbers of returned test books and/or answer documents to the vendor and Agency records. The Agency will be held harmless for any breaches of security that infringes upon the Vendor’s copyright or secure testing requirements.
The vendor will work with the Agency to establish a timeline for processing orphans. Orphan documents will be processed in an expedited manner
188.8.131.52.5 Store/Retrieve Paper Answer Documents and Test Books
The vendor may periodically be required to retrieve answer documents from storage. The vendor will be responsible for costs associated with retrieval and possible delivery of these materials to the Agency. In addition, when errors are found, the vendor may be required to re-score and re-report these documents. The vendor must retain original answer documents.
184.108.40.206.6 Disposition of Paper Material
Upon verification of the individual test booklet/answer document identification numbers of all test books/answer sheets returned by the counties and acceptance by the Agency of accurate results files, the vendor will inventory and store unused paper test books, manuals, and other materials for a period of twelve months. After acceptance by the Agency of accurate computer files, answer documents must be stored for the life of the contract. The vendor may propose a shorter storage time with an explanation for the reduced amount of time.
Test security requirements will be maintained throughout the destruction process.
At the end of the contract, the vendor will ship or destroy the answer documents according to instructions from the Agency, unless the Agency has approved a shorter storage time per the vendor’s recommendation. This destruction will be initiated by a letter from the vendor to the Agency requesting permission to destroy specific materials. Destruction of secure documents must be requested in writing and authorized by the Agency. Further, the vendor must submit a certification of destruction that describes in writing the specific items destroyed. If it is necessary to retain answer documents for longer time periods than previously outlined, the Agency will use additional funds to pay for the storage or request the documents be transferred to Agency for storage at Agency.
220.127.116.11 Processing and Scanning Verification Introduction
The vendor will design and implement the systems required to process and scan the results of student responses from each administration. The vendor will also develop procedures to verify the accuracy of data produced at each processing step. The vendor will work jointly with Agency on finalizing the processing rules and data analyses shortly after award of the contract. The vendor must deliver proposed requirements based on discussions and past performance history to Agency for approval. Example of processing and scanning procedures must be provided for committee review.
Test processing will include receipt of the answer documents, scanning multiple-choice and writing assessment material, and the writing components of the college entrance exam and ensuring the accuracy of data at each processing step. Additional data processing activities may include working with the county and state staff to make corrections and additions to the data files as necessary.
18.104.22.168.1 Processing Specifications
The vendor must complete the development of all data processing, scanning, scoring, and reporting procedures prior to each test administration to ensure all procedures have been checked before the processing of student answer books begins. The vendor must monitor all aspects of the scanning and scoring procedures throughout the entire time actual answer documents are being scanned and scored.
The vendor is responsible for developing processing and scanning verification specifications for the grades 8 and 10 college predictive tests and the college entrance exam that describes in detail all the steps to be implemented to demonstrate and to the Agency that the final reports of results are accurate. The Agency must receive notification and must approve any vendor changes in the processing and scanning verification specifications.
The components of the processing and scanning verification plan include the following.
Verifying total quantities returned by schools
Ensuring all pages are correctly ordered in answer documents
Monitoring intensity levels read by each scanner
Monitoring reading of answer books/answer documents, student bar codes, and other codes identifying the book
Developing guidelines for hand edits
Training staff to perform hand edits
Monitoring hand edits
22.214.171.124.2 Scan Materials
The vendor’s methods of accurate scanning must be verified with each scanner used through the use of periodic recalibration procedures. Scanning must be monitored by the vendor between each scan run and each time a scanner is recalibrated. The plan will identify:
what scanners will be used
how many scanners will be used
the scanner intensity levels to be considered a mark
definition of what marks constitute a double mark
details for monitoring the intensity levels read during scoring, based on analyses of scanner output
The plan must identify what types of monitoring the contractor will be performing and what types of data will be presented to the Agency to verify the scanners are working properly through each scan run of actual scoring. The vendor must provide an example for committee review.
126.96.36.199.3 Materials Edited
Vendors shall provide a description of necessary editing of answer documents and headers which
contain double grids or inaccurate gridding of printed information,
are coded incorrectly with respect to student, school, or county identification, or
are deemed partially or wholly unscorable for some reason.
Vendors will include rules for editing each field on the answer book or answer sheet, noting which willbe flagged, hand-checked, and corrected where necessary, and which fields will rely on scanner discrimination rules and not be hand-edited.
Materials used for training editors will be described and included. The plan will also include a description of the edit reports to be produced to show what books or headers need editing, the outcome of edits, and the procedures for monitoring the edit changes.
188.8.131.52 Disaster Recovery Plan
The vendor shall provide a description of the plan to backup all systems, applications, and databases routinely. Additionally, the vendor shall detail the plan for data recovery in the event a disaster is declared where the data is maintained and stored. Database transaction logs should be archived and maintained online for 48 hours. Back-ups will be maintained offline for the length of the contract.
184.108.40.206 Scoring and Technology Introduction
The proposal should set forth and document the capabilities of the company to score West Virginia materials within the prescribed time limits. The references in the proposal should be able to substantiate these capabilities. The Agency desires to implement scoring processes that are reliable and valid, as well as efficient in terms of time and expenditures.
Vendors must design and implement systems required to process, score, and report the results of student responses from each administration. Therefore, vendors must provide evidence of their ability to assign reliable and valid scores for the methods proposed and also provide a detailed description of how the security of the prompts and student responses will be maintained throughout scoring. Scoring accuracy is a key component in maintaining the quality and integrity of the program while meeting challenging scoring deadlines.
220.127.116.11.1 Develop Specifications
For each administration, the vendor shall provide Agency with a detailed scoring and reporting plan that documents the accuracy of all scoring and report production programs. This plan should detail the process of quality reviews of data and the printed report output during report processing and live report production. Scoring verification steps should include:
developing procedures for the vendor and the Agency to independently verify the calibrating, scaling, and equating
developing procedures for the vendor and the Agency to independently verify the subscores
verifying all items are scored correctly
verifying all reader scores are correctly transferred to the student’s record
verifying the correct number of readers scored each writing sample
verifying the final scores on hand-scored tasks are correctly calculated
verifying all aggregated scores are correctly rounded and reported
arranging for all student, school, county, and state reports generated and proofed by the vendor and Agency
developing procedures and reports to identify duplicate student records within and across counties
detailed plan to create output files
detailed plan to develop and implement a log-in/log-out process for assessment materials
18.104.22.168 Cheating Detection
The vendor must be prepared to regenerate files when errors are identified or cheating is suspected. Vendor must provide procedures for addressing cheating on the proposed assessments.
The writing component of the college entrance exam will be scored according to the guidelines and procedures as already developed by the vendor which use either image-based or artificial intelligence scoring. The vendor must provide evidence that their guidelines and procedures generate reliable and valid scores. Also, the vendor must provide a detailed description of how the security of student responses will be maintained throughout scoring.
22.214.171.124.1 Produce Scoring Specifications
The vendor is expected to incorporate their procedural, design, and implementation requirements for scoring writing assessment essays into written specifications for committee review.
126.96.36.199.2 Conduct Performance Scoring Operations
Scoring performance items requires a series of procedures designed to maintain the reliability and validity of examinee test scores, provide scoring reliability, quality control, and adequate management control. This RFP calls for the vendor to implement these processes and procedures. The vendor must implement the required procedures throughout the entire contract period. The Agency is to be notified at least two months in advance of any changes or enhancements to the process.
188.8.131.52.3 Produce Scoring Materials
The vendor is responsible for producing the scoring materials for the writing assessment component. The vendor must provide a description of the materials used. The vendor will develop a system for cataloging and storing all scoring materials used during the length of the contract unless the Agency has approved a shorter storage time. The vendor may propose a shorter storage time with an explanation for the reduced amount of time.
184.108.40.206.4 Scoring Reports
In the proposal, vendors will list and describe reports used internally to monitor the quality and pace of the scoring session. The Agency requires access to copies of various internal reports generated by the vendor during the training and scoring sessions for committee review.
220.127.116.11.5 Monitor and Maintain Scoring Quality
Monitoring and maintenance procedures are intended to establish and maintain high levels of scoring accuracy. The vendor must outline their system of monitoring and maintaining scoring quality.
18.104.22.168 Scoring Personnel
22.214.171.124.1 Scoring Directors for Image-based Scoring
For image-based scoring, the vendor will assign its most qualified scoring staff to be project director, site scoring director and scoring directors for West Virginia scoring. They must be highly qualified to serve in these positions and must have an appropriate educational background and extensive experience in directing state-level performance scoring projects as members of the vendor’s regular scoring staff. All staff must have worked in their assigned roles for the vendor on a regular, continuing basis. The vendor will appoint a project director to serve as the vendor’s overall director for the project.
126.96.36.199.2 Recruit and Hire Readers
For imaged-based scoring, all readers must have earned a bachelor's degree in Mathematics, Reading, Science, education, or other related field. The vendor must describe their screening process for hiring readers.
188.8.131.52.3 Training and Qualifying of Readers
For imaged-based scoring, the vendor will determine the training and qualifying of the readers for the writing component of the college entrance exam in Grade 11 and for the optional make-up/re-test in grade 12.
At the conclusion of training, qualified readers will be taught how to use the Agency’s alert system to identify students whose responses indicate the need for an outside agency’s intervention.
184.108.40.206.4 Scoring Sites
Writing assessment scoring must be conducted at the vendor’s established scoring sites that draw on the vendor’s most experienced pools of readers who participate in scoring activities on a regular basis throughout the calendar year. The vendor’s proposal will identify the number and locations of proposed scoring sites. The scoring sites must be secure facilities that employ procedures and technology to control access and maintain the security of materials.
At the Agency’s option, observers may be allowed access to the scoring centers for brief periods of time for the purpose of generally understanding the process. Agency official or vendor staff designated by the Agency will accompany such visitors.
220.127.116.11.4.1 Expedite Performance Scoring
As stated, the Agency desires that the tests be scored and reported in a timely manner from the time the vendor receives the tests. The vendor must submit to the Agency a timeline for scoring and reporting outlining when student and aggregate results for schools and counties will be available.
Timelines will be developed for the grades 8 and 10 the college predictive tests, as well as the college entrance exam in grade 11 and the make-up/retest college entrance exam in grade 12.
18.104.22.168.5. Overall Scoring Quality Control
The vendor must provide for quality control systems to verify the accuracy of the scoring, processing, and reporting. In addition, the vendor will provide the results of these quality control reviews to the Agency so that the Agency can ensure any identified problems have been rectified.
The proposal should document the capabilities of the vendor to fulfill West Virginia reports requirements within the prescribed time limits. Vendor procedures and sample report guidelines are found in this section. The vendor’s references (See Section 22.214.171.124, 126.96.36.199, and 188.8.131.52) must be able to substantiate these capabilities.
The vendor shall provide the Agency with a detailed scoring and reporting plan that documents the accuracy of all scoring and report production programs. This plan must detail the process of quality reviews of data and the printed report output during report processing and live report production.
The vendor is expected to incorporate the procedural, design, and implementation requirements for reporting tasks into written specifications. The vendor is to submit to Agency specifications for each standard report for the grades 8 and 10 college predictive tests and the college entrance exam that include
samples of all reports in this proposal
how the data on each report are generated (i.e., which population of students)
when the report is shipped
who receives the report with the number of copies received
184.108.40.206.2 Report Development
The vendor is to describe how the standard reports for the grades 8 and 10 college predictive tests and the college entrance exam are generated as well as identifying the units of analysis for inclusion in the reports. The Agency desires reports by student, class, school, county, and state. At the school, county, and state levels, Agency desires reports that include subgroup results (economically disadvantaged students, students with disabilities, students with limited English proficiency (Limited English Proficient), major racial and ethnic groups and gender). Special education reports will also be provided at the school, county, and state levels. The vendor is encouraged to submit mockups of customized reports to address the reporting needs of the Agency.
Test results will be reported by scale score and percentile, quartile and/or other appropriate test measurements by students for each previously identified subgroup. Subtest results will be reported by standards/strands/skills and subtests when appropriate. The Agency requires an item analysis for each assessment that is administered.
The vendor and Agency will extensively review all data files for grades 8, 10, and 11 before they are used to produce live reports. The vendor must produce a live data file with a sample population composed of three counties selected by the Agency for the grades 8 and 10 college predictive tests. This file will be used to check student-level and aggregated data for each content-area test at each grade. Each phase of reports will be created from this live data check file; both the file and reports should be sent to the Agency for verification and approval.
The Agency will review the data and draft reports, and will work with the vendor to resolve any questions. Agency expects the vendor to conduct an extensive quality check before the file and final reports are sent to the Agency.
220.127.116.11.3 Update Report Designs
The Agency desires easy to understand reports that are both attractive and technically defensible The Agency recognizes that the vendor will review and update the standard reports for the grades 8 and 10 college predictive tests and the college entrance exam. It is expected that the report formats will not change extensively from year to year. When changes or enhancements are made to the standard reports by the vendor, the Agency is to be notified in a timely manner.
The vendor is responsible for annually reviewing and updating the design of the any customized individual student, school, county, and state reports of test results in consultation with the Agency. Though it is expected report formats will not change extensively from year to year, the vendor should, after each administration, pursue reporting requirements from Agency and make any changes required by Agency until final approval is given. No extra cost may be charged to Agency.
Vendors must be prepared to present for the Agency’s consideration creative and innovative customized report designs that take advantage of current technologies for color printing and data merging. The reports must provide numeric, verbal, and graphic presentations of assessment results that effectively communicate with intended audiences, including students, teachers, parents, and the general public.
18.104.22.168.4 Report Delivery
During each administration, numerous reports and data files are provided to students, schools, counties, and the state for students, parents, educators, and the public with data aggregated in various ways. The actual reports and data files to be generated are described in Sections 22.214.171.124 and 126.96.36.199.4.1 The vendor must prepare the data files using formats approved by the Agency.
Requirements are established for many reports to be available as electronic files in formats compliant with Section 508 the Rehabilitation Act (Refer to: http://www.section508.gov/) and to allow the files to be both viewed on a website and downloaded.
The vendor should submit for committee review the delivery schedule for each proposed report.
188.8.131.52.4.1 Report Phases
Following vendor quality checks, reports are to be generated and disseminated according to the timelines approved by the Agency for the grades 8 and 10 college predictive tests, and the grade 11 college entrance exam. The vendor will adhere to the established timeline for the distribution of student and school reports for the grade 12 make-up and re-test for eligible students of the college entrance exam.
The vendor is to submit proposed timelines to the Agency for county and state reports for the college predictive exams in grades 8 and 10, grades 11 and 12 college entrance exam, and the grade 12 make-up/retest college entrance exam.
Reports will be delivered in two phases:
Phase I -- Student Reports.
Student reports for 8 and 10 must be back to school no later than 3 weeks after receipt of the test booklets/answer documents. The vendor may propose Erasure Analysis reports that lists the classrooms and students within those classrooms who have excessive wrong-to-right erasures. Based on the reports, the Agency may require the vendor to void student test scores.
Phase II -- School, County and State Reports
These reports contain all standard and customized school, county, and state aggregations of test results by scale scores and other appropriate test measurements, content, standard/strand/skill when appropriate, and subtest no later than 6 – 8 weeks after test administration. The Agency requires an item analysis for each assessment that is administered. Additionally, the vendor should provide an item analysis for the 11th grade college entrance exam.
All reports for schools are to be packaged by school, but sent to the County Test Coordinators (CTCs) or vendor may propose to send the materials directly to schools. All printed products will be proofed by the vendor and copies will be sent to the Agency for proofing and approval prior to mailing any product to the counties. The Agency must review at least one copy of each report before shipment of any reports. The vendor shall be responsible for maintaining copies of electronic data files for each test administration.
The vendor is to send copies of the school reports and the county report to the CTCs in PDF format on CDs. The vendor is to send the Agency copies of all school and county reports by county in PDF format on CDs.
As part of the process of approving files for the accelerated and regular reporting stream, the Agency reserves the right to request some records be removed from processing until specific issues are resolved. These issues include duplicate records, records with blank WVEIS Student Identification Numbers and/or blank names, schools or students whose test records are under investigation for possible cheating, or other issues that might affect school totals. The issues regarding the suppressed records will be dealt with as soon as possible after reporting is completed.
The vendor may be requested to change the score reported flag on the file to one that would not report the student’s score, pull test documents to resolve duplicate tester issues, add a corrected WVEIS number or corrected name to a record, produce student reports (as directed), and/or lists of students when 10 or more students in a specific school are affected. The vendor will work with the Agency to establish a timeline for the processing and reporting of these records.
184.108.40.206.4.2 Electronic Records
For each administration, the vendor will also supply the Agency with an electronic file, in a format approved by the Agency, containing data aggregated by test for each school, county, and the state. These electronic records will agree with the data reported on summary lines in the county, state, and school level reports. Additional summary statistics for each school and county and the state will be reported by disaggregated characteristics such as racial/ethnic group, gender and other demographic information. Every summary statistic printed in the paper reports should be represented in this file.
The Agency will select data elements from the reports provided to schools and counties, and the vendor will draft a format for this file. The Agency will revise the file format as necessary to accommodate changes in the programs from administration to administration.
The Agency may specify variables and field lengths for all files. Before providing a data file to the Agency, the vendor will be responsible for checking to ensure all files are consistent and accurately reflect the data provided on the reports. The Agency will independently verify the consistency and accuracy of the data files.
220.127.116.11.4.3 Report CDs
The vendor will send all school and county reports to the County Test Coordinator in Adobe® PDF format on a CD.
All customized reports will be generated and distributed to appropriately to students, schools, counties and/or the state according to the timeline presented to the Agency. All reports for schools are packaged by school but are to be sent to the County Test Coordinators.
All printed products will be proofed by the vendor, and copies will be sent to the Agency for proofing and approval prior to mailing any products to the counties. The Agency must review at least one copy of each report before shipment of any reports.
18.104.22.168.6 Reports Descriptions
Vendor must provide a detailed description and an example of each report for the tests proposed in this section. The vendor's proposed timeline for completing proposed services must respect the 2008 – 2009 test administrations dates for the fall administration of the grades 8 and 10 college predictive tests and the grade 11 college entrance exam at a yet to be determined date. All reports are to be separated before being shipped to counties or to Agency. Different reports produced from the same file may not be attached to one another or bound together.
In addition, the vendor is responsible for processing all missing or erroneous reports throughout the duration of the contract. Copies of data files for each test administration shall be maintained throughout the duration of the contract. Three distribution levels will be specified - school, county, and state. School reports will be shipped to the County Test Coordinator at each county central office.
The vendor must list every standard and proposed customized report for the program/grade level assessment and consider providing an item analysis for each test proposed.
22.214.171.124 College Reports
The vendor will submit college entrance exam results to the colleges, universities, and/or agencies as identified by the student. The vendor’s existing guidelines for submitting student college entrance exam results will be honored by the Agency.