Agency requires the vendor’s data management system to interface with the Agency’s data management system. A general information file of individual student performance must be submitted to the Executive Director, Office of Technology, immediately upon completion of scoring not to exceed 3 weeks (for multiple choice tests) or 6 weeks (for multiple choice and constructed response tests).
Agency requests vendors to provide software solutions to the data management and disaggregation of data at the class, grade, school, county and state levels. In addition, disaggregated group reports must be available by the following classifications:
Limited English Proficient
other groups as specified over the life of the contract
School, county, and state level reports to be received will be provided by the vendor through a secure FTP site in the same form as the specified reports. The vendor will work in conjunction with the Agency to finalize the data and data layout. The Agency will have final approval of variable names and the formatting layout. All optional items for purchase will follow the established format.
Any proprietary software required (along with all software support) to read the data must be included for the Agency and updated throughout the contract. Vendors are to describe this software in full. If additional copies will be required at the county level, pricing for this must be included for the life of the contract.
Writing Assessment 3-11
(Contingent upon funding, the Agency reserves the right to activate this section by grade level at any time during the life of the contract)
220.127.116.11 Writing Assessment 3-11
The Agency seeks innovative and technological solutions that will permit the Writing Assessment administration, scoring and reporting processes to be as efficient as possible, thus resulting in shorter turnaround time between testing and reporting. The vendor must develop reliable and valid prompts and must design and implement the systems required to process, score and report the results of student responses from each administration. All student responses shall be scored using the West Virginia Six Point Rubric. (Refer to http://writing.k12.wv.us)
Beginning with the 2007-08 field test, the 2008-09 operational assessment and continuing through the Spring 2013-14 operational assessment, each spring administration of the WESTEST will include a Writing Assessment component for grades 3-11. The Writing Assessment scores for Grades 3-11 will be combined with the Reading/Language Arts scores of WESTEST. The total score will be included in the Annual Yearly Progress (AYP) report for Grades 3-8 and Grade 11. The Grades 9 and 10 Writing Assessment scores will be combined with the Reading/Language Arts scores of WESTEST but will not be included in AYP calculations. Vendors must provide the formula that will be used for combining the Writing Assessment score with the WESTEST Reading/Language Arts score.
The objective of the Agency is to administer the Writing Assessment as late as feasible in the school year without interfering with the administration of the WESTEST. The Writing Assessment may be administered earlier than the Reading/Language Arts component of WESTEST; however, the vendor must provide a combined score of Reading/Language Arts and Writing Assessment to the Agency at the same time and reported as a combined score on the appropriate reports. Vendors must provide evidence of their ability to assign reliable and valid scores for the methods proposed (either human or mechanical) and also provide a detailed description of how the security of the prompts and student responses will be maintained throughout administration and scoring. Scoring accuracy is a key component in maintaining the quality and integrity of the Writing Assessment while meeting challenging scoring deadlines.
Agency requests that optional services and enhancements for Writing Assessment in Grades 3-11 be made available upon requests for improvements toward product efficiency, usability and expanded reporting capabilities. The Agency requests that each vendor provide costs/quotes for any optional services, enhancements or projected updates in the proposal. These optional services must be available upon request by the Agency.
The Agency is requesting proposals on the following options of the Statewide Writing Assessment Program:
TABLE 27: Writing Assessment Options: (Vendors must submit costs for all options)
Description of Option
OPTION # 1
Writing Assessment for grades 3-11 as an online word-processed and computer scored essay based on a prompt.
OPTION # 2
Writing Assessment for grades 3-11 as an online word-processed and computer scored essay based on a prompt with an accompanying reading passage.
OPTION # 3
Writing Assessment for grades 4, 7, 10, and 11 as an online word-processed and computer scored essay and grades 3, 5, 6, 8 and 9 as a handwritten paper-based essay which will be scanned for image-based scoring by human reader based on a prompt.
OPTION # 4
Writing Assessment for grades 4, 7, 10, and 11 as an online word-processed and computer scored essay and grades 3, 5, 6, 8 and 9 as a handwritten paper-based essay which will be scanned for image-based scoring by human reader based on a prompt with an accompanying reading passage.
OPTION # 5
Writing Assessment for grades 4, 7, 10, and 11 as an online word-processed and computer scored essay based on a prompt.
OPTION # 6
Writing Assessment for grades 4, 7, 10 and 11 as an online word-processed and computer scored essay based on a prompt with an accompanying reading passage.
TABLE 28: 2007 – 2008 Field Test Prompt/Passage Specifications
Descriptive, Narrative, Persuasive, & Informative – Prompts with Passages
TABLE 29: Field Test Specifications - Scoring of 2007- 2008 Field Test Essays
NUMBER OF STUDENTS
NUMBER OF READS/ PROMPT/ESSAY
TOTAL NUMBER OF READS/GRADE
18.104.22.168.1 Operational Tests 2008-2009 through 2013-2014
Vendors will be supplying materials to West Virginia’s 55 counties and two special districts, (the School for Deaf and Blind and Institutional Schools), parochial schools, private schools and home schools. At the time of this RFP, approximately 710 public/private/parochial schools are participating in West Virginia testing. Enrollment figures in Table 29 reflect projected enrollment figures (based on current enrollment) for the first operational year. Vendors must provide appropriate quantities of overage for all test materials.
TABLE 30: Grades 3-11 – Operational Test Specifications 2008-2009 THROUGH 2013-2014
TABLE 30: Grades 3-11 – Operational Test Specifications 2008-2009 THROUGH 2013-2014 (Continued)
THINKING SKILL LEVEL
22.214.171.124.1.1 Test Design
All prompts for grade 3-11 must be in alignment with the 21st Century West Virginia Reading Language Arts Content Standards and Objectives. (See Policy 2520.1 at http://wvde.state.wv.us/policies/p2520.1_ne.pdf ) A pool of prompts for all grade levels must be developed and field tested in Spring 2008. All responses will be scored using the West Virginia Six-Point Rubric (See http://writing.k12.wv.us/).
West Virginia is considering two scoring methods.
1. A paper-based, handwritten response to be scanned and scored using an electronic image-based system. Scoring would be scored by human readers with two (2) reads per essay.
2. An online system with the response to be word processed. Student responses would be scored with artificial intelligence scoring with one electronic read and a 10% human read behind. The artificial intelligence scoring engine must be trained with field test essays, written by West Virginia students, and hand-scored using the West Virginia Six-Point Rubric.
If a paper-based, handwritten response is the method of testing, an operational writing answer document will be required. This document will be at the minimum a four-page document, consisting of a machine scannable front cover with appropriate information (e.g., title, name, birthdates, grade, logo, date, school name, county name, WVEIS number). Remaining pages will include machine scannable preslugged information, the Writing Prompt/Passage, Directions, a Writing Checklist, and a sufficient number of lined pages for final draft writing. Lined writing paper for rough draft writing will be provided for students by the school. For the field test, answer documents with alternating prompts/passages will be packaged in spiraled order by grade level. For the operational assessment, answer documents will be arranged by randomly assigned alternating prompts/passages by grade level. It is expected that the response will be scanned and scored using image-based scoring by human readers with two reads per essay.
If an online word-processed response using artificial intelligence to score student responses is the method of testing, the vendor must use an intelligent scoring system that emulates the process carried out by human scorers. The system must be theoretically grounded in the traditions of cognitive processing, computational linguistics and classification. The system must be “trained” with a set of previously scored responses containing “known score” marker papers for each score point. These field test papers, written by West Virginia students and hand scored using the West Virginia Six-point Rubric, will be used as a basis for the system to utilize the rubric and the pooled judgments of the human scorers. The system must internalize the characteristics of the responses associated with each score point and apply this intelligence in subsequent scoring. Sufficient student responses must be used to calibrate the system. Each response used will have been scored by at least one expert hand scorer using the West Virginia Six-Point Rubric.
Enrollment figures reflect current accurate enrollment for the field test and for the first operational year. Vendors must supply appropriate quantities of overage for all test materials. (See Table 29)
Six broad options for Writing Assessment are listed in Table 27. These Options will be referred to throughout the RFP. Vendors must submit cost proposals on all options and all tasks associated with that option.
126.96.36.199.1.2 Field Test
One year prior to the 2008–2009 operational assessment, the vendor will provide a pool of prompts for field testing that will provide a pool of quality prompts for selecting operational prompts for the next six operational years. Field testing will be administered in Spring 2008. Field testing will result in at least 2 usable prompts for each genre for each grade level for the six-year life of the contract. Field testing will be conducted using the option selected by the Agency. Materials will be packaged with prompts in alternating (spiraled) order and distributed by the administration vendor using the same paper or online layout that will be used for the operational assessment. West Virginia estimates that at least 1,000 student responses per prompt, to include students from all subgroups, will be needed; however, vendor may propose with justification a different number of student responses per prompt.
Field test papers will be scored by vendor staff and/or West Virginia teachers. The statistics from field test scoring will be used to select operational prompts for the next six years. For the field test, vendors will be supplying materials to West Virginia’s 55 school districts and two special districts -- the School for Deaf and Blind and Institutional Schools. At the time of this RFP, approximately 710 public/private/parochial schools are participating in West Virginia testing. Enrollment figures in Table 29 reflect projected enrollment figures with a 5% overage for the field test and the first operational year. Vendors must provide appropriate quantities of overage for all test materials.