Request for Proposal (rfp)



Download 5.9 Mb.
Page7/38
Date09.08.2018
Size5.9 Mb.
TypeRequest
1   2   3   4   5   6   7   8   9   10   ...   38

3.2.2.1.22.3 Reporting


Any and all reports requested by the paper and pencil exam must be replicated, but not be limited to those requested by the assessment system in general. The basic reports names are (for detailed descriptions See Section 3.2.1.23.1):

  • STUDENT REPORT

  • INDIVIDUAL ITEM ANALYSIS REPORT

  • ITEM ANALYSIS SUMMARY BY SUBGROUP REPORT

  • CONFIDENTIAL ROSTER REPORT

  • CONFIDENTIAL SUMMARY REPORTS

  • GENERAL RESEARCH FILE

  • STUDENT LABEL

  • ELECTRONIC DATABASE PROGRAM OF STUDENT PERFORMANCE



3.2.2.1.22.4 Disaster Recovery Plan


The vendor shall provide a description for committee review of the plan to backup all systems, applications, and databases routinely. Additionally, the vendor shall detail the plan for data recovery in the event a disaster is declared where the data is maintained and stored. Database transaction logs should be archived and maintained online for 48 hours. Back-ups will be maintained offline for the length of the contract.
3.2.2.1.22.5 FTP Site

Vendors must include provisions for a secure File Transfer Protocol site for data, documentation and report transfer from the vendor to the Agency and from Agency to the vendor. This site must be maintained by the vendor who will ensure its continued availability throughout the life of the program.


3.2.2.1.23 Reporting

The Agency desires easy to understand reports that are creative, attractive and technically defensible. Vendors must present innovative report designs that take advantage of current technologies for color printing and data merging. Reports should look similar to the WESTEST reports. (See http://westest.k12.wv.us/reports.htm.) The reports must provide numeric, verbal, and graphic presentations of assessment results that effectively communicate with intended audiences, including students, teachers, parents, and the general public.


At a minimum, the vendor must supply the reports listed in the Report Descriptions (See Section 3.2.2.1.23.1). The proposal should document the capabilities of the vendor to fulfill West Virginia reports requirements within the prescribed time limits. Vendor procedures and report guidelines are found in this section. The vendor’s references (Section 3.2.4.14) must be able to substantiate these capabilities.
The vendor shall provide the Agency with a detailed scoring and reporting plan that documents the accuracy of all scoring and report production programs. This plan should detail the process of quality reviews of data and the printed report output during report processing and live report production.
Minor adjustments to the reports should be anticipated by the successful vendor. In responding to the reporting requirements, vendors are encouraged to suggest combinations of report formats, or innovative graphic or numeric displays.
3.2.2.1.23.1 Reports Descriptions

Vendor's proposed timeline for completing proposed services must respect the Spring 2008 field test and 2009 operational test administration dates. All reports are to be separated before being shipped to counties, or the Agency.


In addition, the vendor will be prepared to process missing or erroneous reports throughout the duration of the contract. Copies of data files for each test administration shall be maintained throughout the duration of the contract. Three distribution levels will be specified - school, county, and state. School reports will be shipped to the County Test Coordinator at each county office. The WESTEST reports for the program must at a minimum include:

1. STUDENT REPORT:

The individual Student Report will at a minimum include the following:



    • Student name, grade, date of birth, WVEIS #, class, school, county, state, and explanatory information about the scores

    • Total summative score and performance level

      • Writing performance, based on the writing prompt, for all students of the same grade level must be reliability combined into each student’s Reading/Language Arts score

    • Definition of terms

    • Performance level descriptors

      • One side of student report will capture the performance level descriptors

    • Lexile/Quantile scores


2. INDIVIDUAL ITEM ANALYSIS (Per Student):

The Individual Item Analysis shall at a minimum provide for each student



  • Listing of all content standards and objectives for all items on the test

    • All objectives must be clustered under appropriate standards

  • Indication response to the item is correct, indication response to the item is not correct for multiple choice items

  • Indication of the number of points possible, points received by student for constructed response items

  • Thinking skills levels by item/definition of thinking skills

  • Definitions for all unfamiliar terms on the report


3. ITEM ANALYSIS SUMMARY BY SUBGROUP REPORT (School, County and State)

The Item Analysis Summary by subgroup report shall be organized by grade by school, by grade by county, and by county by state and shall include at a minimum the following:



  • Results disaggregated by All, low Socio-economic Status, Special Education, Black, White, Hispanic, Asian/Pacific, Native American/Alaskan, Limited English Proficient, for grades K – 11

  • Content area, item number, CSO number and description

    • All objectives must be clustered by standards for each content area

  • Percent of students with item correct, percent of students with each score point, number of points possible, thinking skills levels by item

  • Definition of thinking skills level

  • Definition of all unfamiliar terms on the report


4. CONFIDENTIAL ROSTER REPORT (ALPHA Level by Grade for School Level Only):

The Confidential Roster Report shall be organized by grade by school and by grade by county and shall include the following:



  • Student names in alphabetical order by grade (last name, first name, middle initial), grade, date of birth, WVEIS number, school, county, test date, prompt type

  • Student scale scores for all content areas and performance levels, lexiles and quantiles

  • All content areas should have assessment scores and performance levels

    • Show combined Reading/language Arts/Writing Assessment scores and separate writing score and performance level


5. CONFIDENTIAL SUMMARY REPORTS (School, County and State):

The West Virginia Confidential Summary Report shall be prepared for all schools, counties, and state. They shall contain a graph of the percent of students who attained each performance level category. The report shall also show the distribution of analytic trait scores for the group. The report shall be organized by grade by school, by grade by county, and by grade by state and shall at a minimum include the following:



  • Grade and test date

  • Number of students tested by content areas and subgroups (all, gender, race/ethnicity, students with disabilities, Limited English Proficient students, migrant, economically disadvantaged)

  • Performance levels by aggregate number and percent of students at each performance level

  • Mean scale scores and grade level mastery of the content area

  • Number of students tested by content standard, grade level mastery of content standards, and mean percent correct by content standard

  • Definition of all unfamiliar terms on the report

  • Writing performance levels for all students of the same grade level must be reliability combined into the Reading/Language Arts score

  • Number of students tested by content standard, grade level mastery of content standards, and mean percent correct

  • Definition of mean percent correct

Upon all corrections being made with vendor and Agency, the vendor will supply

1) all corrected reports to Agency via an FTP site within 8 weeks of receipt and scoring of assessments and

2) the aggregate performance, as per the guidelines of this Confidential Summary Report, of all grade levels within the each school, each county and the state.


6. GENERAL RESEARCH FILE:

A report shall be programmed and made available to provide electronic data for preparing accountability reports. This file shall be organized by school, grade, county, and state, subgroups, and shall agree with the data reported on summary lines in the school, county, and state level reports.


7. STUDENT LABEL

    • The Student Label will at a minimum include the following:

      • Student name, grade, school, test date, gender, date of birth, WVEIS number, content area, scale score, performance level, prompt type by grade by school

      • Self-adhesive to allow attachment to the student record

8. ELECTRONIC DATABASE PROGRAM OF STUDENT PERFORMANCE



    • Individual Student Performance with a re-rostering option for teachers

      • Reports by subgroup to include Title I and Special Education

and all other federally required subgroups

      • Capability of producing longitudinal data reports

      • Program should be customized to address the reports required in this proposal

      • Product may be software or web-based

      • Product should be compatible with software programs

Any proprietary software required (along with all software support) to read the data must be included for the Agency and updated throughout the contract. Vendors are to describe this software in full. If additional copies will be required at the county level, pricing for this must be included for the life of the contract.


This section requires the vendor to provide costs for placing all reports except the student report on a secure FTP site for local school districts for electronic retrieval and printing. All individual student reports are to be printed by the vendor and distributed to county test coordinators for dissemination to local schools. The vendor must supply two copies of each individual student report to the local school district.
3.2.2.1.23.2 Develop Specifications

The vendor is responsible for developing specifications for each administration that describes in detail all the steps to be implemented to demonstrate to the Agency the final reports of results are accurate. The vendor is expected to incorporate the procedural, design, and implementation requirements for reporting tasks into written specifications initially developed by September 2007 for the Spring 2008 field test administrations. The vendor must produce final specifications and mockups of proposed report forms for each following administration within a similar timeline.


The vendor is responsible for drafting specifications for each report that include:

  • a description of the report

  • how the data on each report are generated (i.e., which population of students)

  • in which shipment the report is included

  • who receives the report with the number of copies received

  • a sample of the report

This plan should detail the process of electronic quality reviews for the data and the printed report output during report processing and live report production. The vendor will work jointly with the Agency on finalizing the scoring rules and final reporting considerations shortly after award.


3.2.2.1.23.3 Report Development

The units of analysis for inclusion in West Virginia reports are the student, class, school, county, and state. At the school, county, and state levels, reports will include subgroup results (economically disadvantaged students, students with disabilities, students with limited English proficiency (LEP), major racial and ethnic groups and gender) Special education reports will also be provided at the school, county, and state levels.


Test results will be reported by scale score and achievement level. Subtest results will be reported by standard and subtests when appropriate (e.g., usage/mechanics, rhetorical skills, pre-algebra and geometry). Also, test results by scale score and achievement level for the previously mentioned subgroups will be reported. The tests will be placed on a vertical scale, but percentile scores are not reported.
The Lexile Framework® for Reading is a scientific approach to measuring reading ability and reading materials. A Lexile measure represents both the difficulty of a text, such as a book or article, and an individual’s reading ability. The Lexile scale is a developmental scale for measuring reader ability and text difficulty ranging from below 200L for beginning readers and beginning-reader materials to above 1700L for advanced readers and materials. Knowing the Lexile measure of a reader and the Lexile measure of a text helps to predict how the text matches the reader’s ability—whether the text may be too easy, too difficult or appropriate. All Lexile products, tools and services rely on the Lexile measure and the Lexile scale to match reader and text. (Refer to www.lexile.com)
The Lexile measure should be linked to the reading or reading comprehension scale score. For instructional purposes, the Lexile measure should be reported at the student and classroom levels. Lexile measures should appear on reports that are sent home to parents and reports that are provided to the current grade-level teacher and/or the next grade-level teacher.
The Quantile Framework® for Mathematics is a scientific approach to measuring mathematical achievement and concept/application solvability. A Quantile measure represents the difficulty of a mathematical skill, concept or application (called a QTaxon) and a developing mathematician’s mastery of the QTaxons in the areas of geometry, measurement, numbers and operations, algebra, and data analysis and probability. The Quantile Framework spans the developmental continuum from kindergarten mathematics through the content typically taught in Algebra II, Geometry, Trigonometry and Pre-calculus, from below 0Q (Emerging Mathematician) to above 1400Q. Quantile measures take the guesswork out of determining which mathematical skills a developing mathematician has mastered and which ones require further instruction. (Refer to www.quantiles.com)
The Quantile measure should be linked to the mathematics, mathematics application, or problem solving scale score. For instructional purposes, the Quantile measure should be reported at the student and classroom levels. Quantile measures should appear on reports that are sent home to parents and reports that are provided to the current grade-level teacher and/or the next grade-level teacher.
The vendor and the Agency will extensively review all data files before they are used to produce live reports. The vendor must produce a live data file with a sample population composed of three counties selected by the Agency. This file will be used to check student-level and aggregated data for each content area test at each grade. Each phase of reports will be created from this live data check file; both the file and reports should be sent to Agency for verification and approval.
Agency will review the data and draft reports, and will work with the vendor to resolve any questions. Agency expects the vendor to conduct an extensive quality check before the file and final reports are sent to Agency.
3.2.2.1.23.4 Update Report Designs

The vendor is responsible for annually reviewing and updating the design of the individual student, school, county, and state reports of test results in consultation with the Agency. Though it is expected report formats will not change extensively from year to year, the vendor should, after each administration, pursue reporting requirements from Agency and make any changes required by the Agency until final approval is given. No extra cost will be charged to the Agency.


3.2.2.1.24 Report Delivery

During each administration, numerous reports and data files are provided to students, schools, counties, the state, and the general public with data aggregated in various ways. The vendor must prepare the data files using formats approved by the Agency.


Requirements are established for many reports to be available as electronic files in formats compliant with Section 508 of the Rehabilitation Act (Refer to: http://www.section508.gov/) and to allow the files to be both viewed on a website and downloaded file.
3.2.2.1.24.1 Report Timelines

The vendor will not provide individual student data or reports for the field test administration.


Following vendor quality checks, reports for the spring test administration must be delivered to the Agency and counties via secure FTP. If a multiple choice option is selected the tests must be scored within three weeks and results must be posted/report within two weeks and if a constructed response option is selected the responses must be scored within six weeks after the vendor receives the tests and all student reports should be shipped to local counties via hardcopy within eight weeks after the vendor receives the tests.
All student reports for schools are to be packaged by school name, but sent to the County Test Coordinators. All printed products will be proofed by the vendor and copies will be sent to the Agency for proofing and approval prior to mailing any product to the counties. The Agency must review each report before shipment of any student reports. All student reports should be original laser copies.
The Agency reserves the right to request some records be removed from processing until specific issues are resolved. These issues include duplicate records, records with blank WVEIS numbers and/or blank names, schools, or students whose test records are under investigation for possible cheating, or other issues that might affect school totals. The issues regarding the suppressed records will be dealt with as soon as possible after reporting is completed.
The vendor may be requested to change the score reported flag on the file to one that would not report the student’s score, pull test documents to resolve duplicate tester issues, add a corrected Student Identification Number or corrected name to a record, produce Student Reports (as directed), and/or School Lists of Students. The vendor will work with the Agency to establish a timeline for the processing and reporting of these records.
3.2.2.1.24.2 Electronic Records

For each administration, the vendor will supply the Agency with an electronic file, in a format approved by the Agency, containing individual student data aggregated by grade and subject for each school, county, and the state. These electronic records will agree with the data reported on summary lines in the county, state, and school level reports. Additional summary statistics for each school, county, and the state will be reported by disaggregated characteristics such as racial/ethnic group, gender and other demographic information. Every summary statistic printed in the paper reports should be represented in this file.


The vendor will be responsible for checking to ensure that all files are consistent and accurately reflect the data provided on the reports. The Agency will independently verify the consistency and accuracy of the data files.
3.2.2.1.24.3 Optional Reporting Services

The Agency requests that each vendor provide costs/quotes for any optional services, enhancements or projected updates in the proposal. Please provide information about product efficiency, usability, expanded reporting capabilities and costing. These optional services must be available upon request by the Agency.


3.2.2.1.25 Data Management


The Agency requires the vendor’s data management system interfaces with the Agency’s data management system. A general information file of individual student performance must be submitted to the Executive Director, Office of Technology, immediately upon completion of scoring. Please note that the vendor has three weeks to score multiple choice test options and six weeks to score tests with constructed response items.
Agency requires vendors to provide software solutions to the data management and disaggregation of data at the class, grade, school, county and state levels. In addition, disaggregated group reports must be available by:

  • Limited English Proficient

  • race/ethnicity (as specified in NCLB legislation)

  • gender

  • economically disadvantaged

  • migrant

  • students with disabilities

  • other groups as specified over the life of the contract

School, county, and state level reports will be provided by the vendor through a secure FTP site in the same form as the specified reports. The vendor will work in conjunction with the Agency to finalize the data and data layout. The Agency will have final approval of variable names and the formatting layout. All optional items for purchase will follow the established format.


Any proprietary software required (along with all software support) to read the data must be included for the Agency and updated throughout the contract. Vendors are to describe this software in full. If additional copies will be required at the county level, pricing for this must be included for the life of the contract.
3.2.2.1.26 Disposal/Final Delivery/Destruction of Materials

The vendor agrees to deliver to the Agency, or destroy, upon request, all materials and products in all forms developed for and used in conjunction with this project within 30 days following acceptance by the Agency of the final report for the project, including:



  • test items and performance tasks

  • graphics

  • scoring materials

  • test books

  • answer documents

  • final electronic files of ancillary materials

  • computer discs, CDs, DVDs, or other media

  • computer listings

  • computer files

  • paper files

Payment of the final project invoice will not be made until all materials and certification of destruction, as appropriate, are received and approved by the Agency and final payment resolution is agreed to by both parties. Written verification of the delivery or destruction will be provided to the Agency as part of the final contract report.


3.2.2.1.27 Transition Plan

At the conclusion of the program with the current vendor WESTEST Grades 3-8 and 10 (either through the successful completion of the contract period or through termination), it is expected the vendor will directly and fully participate in the transfer of the program to another vendor. Several steps will be involved, and the vendor of the program associated with this RFP is expected to take a leadership role.

Vendors are expected to provide a plan describing, at a minimum:


  • Duration of transition

  • Transition Meetings (see below)

  • Team members designated

  • FTE required by person

  • Determine core transition team

  • Establishment of common definitions for terms

  • Documentation of all meetings and agreements in detail

  • Determine contractor responsibility for each aspect of important deliverables

  • Documentation of all transferables as sent to new vendor and new vendor’s inspection of all materials

  • Documentation/identification of all psychometric files and like documents to be transferred

  • Monitoring timely delivery of agreed transferables

  • Plan appropriate replication studies involving Agency Technical Advisory Committee - Equating and handscoring studies are a minimum

Vendors’ proposals should address completing a full plan for transition prior to the conclusion of 2008. This plan will be updated annually to reflect new information.


3.2.2.1.27.1 Transition Activities

A series of meetings is to be expected to execute transition tasks, as seen in Table 15. Some, or all, of these meetings may take place at the same time and meeting place, based on the vendor’s recommendations.


In the event the vendor is an entity other than the current vendor, the vendor shall conduct transitional activities in accordance with the requirements listed below.


  • By no later than twenty (20) calendar days following the vendor’s initial meeting with the Agency, the vendor must develop and submit a written Transition Plan to the Agency. At a minimum, the Transition Plan must include a schedule for the transition of the WESTEST assessments from the current vendor to the new vendor. The schedule should present a sequential step-by-step description of the tasks, or events, and a timeline for the transition of the WESTEST assessment materials and procedures.

  • The vendor shall agree and understand that the Agency shall have complete and total approval authority over the Transition Plan and shall have the right to require changes, as deemed necessary in order to ensure optimum efficiency.

  • The vendor shall facilitate the transfer of existing Reading/Language Arts, Mathematics, Social Studies and Science custom-developed items and any associated artwork, or passages, from the current vendor.

The Agency anticipates, but cannot guarantee, that items will be available for transfer as follows:



  • Graphics and text for all transferred items will be in Adobe PDF format. All transferred items which have been field-tested will be accompanied by field test statistics. The contractor shall acquire and/or maintain necessary permission for use of all artwork, passages, or other resources associated with custom-developed items.

  • If the nationally-normed component of the WESTEST assessment is other than the TerraNova the contractor shall conduct a technically sound study equating selected-response items of the TerraNova portion of the assessment (currently used in WESTEST assessments) to the nationally-normed assessment provided by the vendor. The vendor shall also equate the previous WESTEST scale scores and national percentile ranks to the new score scales with minimal equating error. The vendor must also ensure alignment of the nationally-normed component of the assessment to the 21st Century WV CSOs. Upon completion, the vendor shall submit a copy of the results of such studies to the Agency. Furthermore, the vendor must provide the Agency with technical evidence that the change in the nationally-normed component of the WESTEST assessment has not threatened the integrity of WESTEST longitudinal data.

  • The contractor shall have completed the transition of the WESTEST to the new contractor by no later than October 1, 2007, pursuant to the Agency approved Transition Plan.


3.2.2.1.27.2 Transitional Meetings

If the contract is awarded to the current vendor, the vendor must develop and submit a written plan to the Agency detailing the management of the work responsibilities during the time in which the previous current contract and the newly awarded contract will overlap.


The contractor shall agree and understand that the Agency shall have complete and total approval authority over the management plan and shall have the right to require changes, as deemed necessary in order to ensure optimum efficiency.
Table 15: Transition Meetings




Meeting Number

Meeting Purpose

1

General orientation of staff (Agency, vendor, and “new” vendor) and goals for transition

2

Discuss Publications and Identification of Transferable Material and Files

3

Discuss Data Collection, Scanning, and Processing

4

Discuss Analysis and Reporting

5

Discuss Master Schedule

6

Discuss Psychometrics, including replication study

7

Discuss Handscoring


3.2.2.1.27.3 Transition Psychometric Issues

Vendors should define in their proposals a set of key psychometric issues to be considered at the time of transition. At an absolute minimum, equating and handscoring replication studies will be included. An exploration should be conducted (and defined here) to determine whether change (positive/negative) in test results is solely related to student performance and not change(s) in testing methodology/material from vendor to vendor. The final Transition Plan will include a literature review detailing past efforts in transitioning programs with a series of recommendations.


3.2.2.1.27.4 Ownership

As stated, all materials and products in all forms developed for and used in conjunction with this project, including test items and performance items, scoring materials, test books, answer documents, final electronic files of ancillary materials, and computer files shall remain the property of the Agency in all phases of transition.



Alternate Performance Task Assessment for Grades 3-8 and 11

(Contingent upon funding, the Agency reserves the right to activate this section by grade level at any time during the life of the contract)
3.2.2.2 Alternate Performance Task Assessment for Grades 3-8 &11Introduction

The West Virginia Alternate Performance Task Assessment (APTA) will be created for the severe and profound special education population in order to meet the requirements of the NCLB legislation. The vendor will work in conjunction with Agency to develop Alternate Performance Task Assessment forms of WESTEST in grades 3-8 and 11 for Spring 2008 field test and Spring 2009 - 2014 operational forms. Two forms will be created for Mathematics, Reading/Language Arts, Science, and Social Studies at each grade level.


APTA measures skills associated with West Virginia’s Extended Content Standards in Reading/Language Arts, Mathematics, Science and Social Studies. The Extended Content Standards are on grade level and have been developed from the general education content standards for each grade, grades 3-8 and 11. These standards were written to be directly aligned and linked to the on-grade 21st Century WV CSOs and are further described via performance level descriptors. (See Policy 2520.16 http://wvde.state.wv.us/policies/p2520.16-math.pdf http://wvde.state.wv.us/policies/p2520.16-reading.pdf)


  • Alternate Performance Task Assessment grades 3-8 and 11 tests must have CRT customized Reading/Language Arts, Mathematics and Science multiple choice (MC) and constructed response (CR) items.


3.2.2.2.1 Item/Form Development Introduction

WV requires the vendor provide a detailed plan of how they propose to meet all development requirements. The following section outlines the required development tasks. Vendors should elaborate on these where appropriate and provide new ideas regarding the development process when possible.


All elements of the development process including procedures, processes, and products used by the vendor to complete contract work are subject to final approval by Agency. Agency is to participate fully in all form construction. Any items developed specifically for Agency will be owned by the Agency. The vendor will work closely with the Agency and groups of West Virginia educators to complete all work tasks. Throughout the contract period, the vendor will confer with the Agency on a continuing and consistent basis and will be involved in frequent face-to-face meetings with the Agency as needed.



Download 5.9 Mb.

Share with your friends:
1   2   3   4   5   6   7   8   9   10   ...   38




The database is protected by copyright ©sckool.org 2020
send message

    Main page