Request for Proposal (rfp)


General Requirements for Algebra I Grades 7-12 Introduction



Download 5.9 Mb.
Page15/38
Date09.08.2018
Size5.9 Mb.
TypeRequest
1   ...   11   12   13   14   15   16   17   18   ...   38

3.2.2.5 General Requirements for Algebra I Grades 7-12 Introduction


The vendor will develop and deliver an online Algebra I End of Course (EOC) Exam to include an online field/pilot test using the selected platform, reports and related technology as outlined in this document. This exam must be aligned to the 21st Century Content and Objectives (CSOs) for Algebra I. The 21st Century Content Standards and Objectives for Algebra I (Refer to http://wvde.state.wv.us/policies/p2520.2_ne.pdf) may be found at http://wvde.state.wv.us/policies/csos/html. The exam (consisting of 65, or fewer, items) must be available for online administration. The exams are to be summative and are to be administered to the student during the final week of the Algebra I class.
The Agency requires two equated operational forms of comparable difficulty with test items being field tested in 2007-2008 and operational test administration beginning at the end of 1st semester 2008-2009. With the operational testing, the two equated forms of the Algebra I EOC will rotate on a semester basis. Due to the Algebra I course-taking patterns, the assessment may be administered beginning in grade 7 and continuing through grade 12.
Agency, in conjunction with the vendor, will determine the mastery level for student achievement.
3.2.2.5.1 Item/Form Development

The Agency requires the vendor to provide a detailed plan of how they propose to meet all development requirements. The following sections outline the required development tasks. Vendors must elaborate on these where appropriate and provide new ideas regarding the development process when possible.


All elements of the development process including procedures, processes, and products used by the vendor to complete contract work are subject to final approval by the Agency. The Agency is to participate fully in all form construction. Any items developed specifically for the Agency will be owned by the Agency. The vendor will work closely with the Agency and groups of West Virginia educators to complete all work tasks. Throughout the contract period, the vendor will confer with the Agency on a continuing and consistent basis and will be involved in frequent Face-to-Face meetings with the Agency as needed.

3.2.2.5.1.1Test Construction Specifics


The specifics for online test construction for the Algebra I EOC Exam must include, but not be limited to, the following:

  • Evidence that test forms align to the test blueprints (21st Century WV CSOs for Algebra I, and Webb’s alignment model for Depth of Knowledge Levels (DOK) (See Section 3.1.2)

  • Test form format that follows the Principles of Universal Design for assessments

  • Test administrator control that allows teachers to set/adjust time limits for test administration in the case of students with disabilities, or in other cases as needed

  • Ability to disable the “time-out” online keystroke feature

  • Schedule of reviews for content and bias

  • Description of process used to flag items based on DIF-analysis and what happens to these items

  • Description of the process to allow only the use of items within a .2–.8 p-value range

Within the proposal, the vendor must provide screen captures of the online test layout and screen shots of the online test environment for committee review.



3.2.2.5.2 Blueprint/Item Specifications

Beginning in 2007, the vendor will work with the Agency to plan a system for accomplishing the task of blueprint and item specification development. The vendor will develop, in conjunction with the Agency, a blueprint (CSO clusters, and DOK Levels) for the Algebra I EOC Exam. The vendor must describe their process for developing and updating the test blueprints and item specifications over the life of the program. The Agency has the final approval on all test blueprints and item specifications and higher order thinking skills. The Agency will expect the vendors to align items to the final DOK to each standard and objective per grade level and per content. All DOK determinations will use the Webb alignment model and definitions of DOKs. The successful vendors will be provided with the DOK distribution charts for the 21st Century West Virginia Content Standards and Objectives by grade level per content area. The newly developed assessments will require a large number of higher thinking skill levels; therefore, all proposals must demonstrate higher levels of thinking.


The vendor must describe their process for developing and updating the test blueprints and item specifications over the life of the contract.
Table 37: Test Design for Algebra I EOC Operational Forms


# Required Items Field Test 2007-2008

# Required Items for Operational 2009-14

378

63

The test items must provide coverage of all of the 21st Century Algebra I CSOs which are to be clustered into at least three areas: Patterns/Relations/Functions, Algebraic Expressions and Equations, and Quantitative Relations. The alignment of the CSOs using the Webb alignment model encompasses the four thinking skill levels. Level 1 thinking skills address the knowledge and recall type questions. Level 2 will address the comprehension, application and conceptual thinking type questions. Level 3 addresses analysis and strategic thinking and Level 4 addresses the synthesis evaluation that uses extended thinking skills. For information about the Webb Alignment Model, refer to Section 3.1.2.


The test design document will describe the relative emphasis of the CSOs and DOK Levels to be assessed. The document will specify the approximate number of items to be included on each form of the assessment. The test design will also describe the measurement model and procedures implemented for the online Algebra I EOC Exam. This blueprint document will form the basis for developing the first draft of the item specifications and the scoring specification documents. The vendor will provide a detailed explanation of their process for creating test blueprints within the proposal.
3.2.2.5.2.1 Item Specifications

The vendor will provide an example of the test blueprints within this proposal for committee review. After the contract is awarded, the vendor will conduct a review of the Algebra I EOC Exam item specifications. The review will be attended by the Agency staff, and may involve West Virginia educators. Based on the review, the vendor will provide and distribute an updated version of the specifications for the exams. It will be the vendor’s responsibility to maintain an updateable electronic content/form management system which contains all item specifications that will be used throughout the development cycle by item writers and reviewers and at item review meetings with West Virginia educators and Agency staff. The vendor will share this secure system with the Agency and the system shall be available twenty-four hours per day, seven days per week. The vendor will provide detailed information and screen shots of this electronic system.


3.2.2.5.3 Item Development/Selection Process

The vendor’s proposal will describe and specify the technical quality of their test development procedures in terms of their alignment to the validity and reliability of testing principles as per the APA Standards. The test development procedures must be submitted for committee review to result in items created to generate reliable and valid performance. During development, the consideration of additional technical factors should include, but are not limited to: test bias and fairness of items for all students; readability of test directions and of test items; and item analysis. For submittal of test items for committee review see Section 3.2.2.5.9.7.


The vendor will describe in detail the procedures to be used in detecting item bias and in ensuring that other technical quality principles are included. The determination of whether, or not, each field tested item goes into the item pool to become eligible for use in operational tests is to be a joint decision between the vendor and the Agency with the Agency having final approval. Agency plans to participate fully in the item selection process and in form construction.
The Algebra I EOC Exam items will measure the knowledge and skills identified in the Algebra I 21st Century West Virginia CSOs. All items and passages must be field tested in an online format prior to actual online administration. The vendor must provide screen captures for review and evaluation for this proposal.
The test format, directions for administering the test, and item development will use the following guidelines:

  • All test materials/answer documents must be formatted in a manner assuring ease of readability. The style of type and font size used shall be large enough to be easily readable in an online environment and have physical characteristics that research has indicated facilitate reading.

  • Verbal information in the item should be appropriate in vocabulary and general readability for the grade level tested.

  • The operational exams will be administered during a two-week window at the end of the Algebra I course

  • Two equated forms of the test will be constructed.

  • The test must have 65 or fewer items, which are divided into two sections and will span a 45-minute block of time each.

After the field/pilot test, the vendor will review appropriate item statistics with select Agency staff. In the proposal, the vendors will describe in writing for the Agency, and for later committee reference, the type of item statistics to be used in determining appropriateness of student performance data for use in final test forms. The vendor will provide quality assurances that students with disabilities, limited English proficiency, and each of the West Virginia NCLB race/ethnic subgroups were appropriately included.


3.2.2.5.3.1 Universal Design Principles

The implementation of Universal Design Principles is a critical feature of the NCLB legislation. The vendor must describe in detail how Principles of Universal Design will be implemented in the creation of items, forms, and all other student assessment materials. The vendor must provide protocols and procedures that will be used for and during all committee review meetings. The vendor must submit screen captures to provide evidence of how the Universal Design Principles were incorporated within the design and development process. The vendor must submit evidence of how the Universal Design elements have been incorporated into their items and form development.



3.2.2.5.4 Reviews

The vendor will be responsible for coordinating and conducting the Content Review Committee meetings and the Bias Review Committee meetings in West Virginia. The Face-to-Face Review will be performed at the vendor’s site, or at a location mutually agreeable to both Agency and the vendor. Meeting plans must include activities, timelines, number of participants required for conducting these review meetings, and the anticipated number of days for review completion. The vendor must submit protocols and procedures that will be followed for the Content Review, the Bias Review, and the Face-to-Face committee meetings. Committee members must have access to computers in order to experience the test passages and items in an online format. The Agency will provide the computers for accessing testing materials. The vendor will have appropriate staff on-site to successfully conduct all review meetings.


The vendor will be responsible for all costs and arrangements related to the review meetings for vendor personnel and meeting materials. The Agency will assume the costs for the facility, refreshments, breakfast and/or lunch, and the travel reimbursements (hotel, mileage, and meals) for participants.
3.2.2.5.4.1 Content Review Meetings

The vendor will outline a plan for a review of test passages and items/prompts and their scoring rubrics by the West Virginia Content Review Committee. The plan will include content reviews of all items with particular emphasis on: the congruency of items with test specifications, readability requirements, Universal Design Principles, technical quality assurances, CSOs and DOK Levels. All proposals must provide examples of vendor protocols and procedures for committee review.


A Content Review Committee will be established to ensure that test passages and items accurately reflect item specifications and test blueprints. The review procedures and materials for orienting reviewers must be described in detail within the proposal. The vendor will provide a minimum of one qualified item writer to facilitate the Algebra I EOC reviews. The vendor will be responsible for preparing all meeting minutes and for providing monthly reports to the Agency.
3.2.2.5.4.2 Bias Review Meetings

The vendor will be responsible for providing a plan for a bias/sensitivity review of test passages and items/prompts and their scoring rubrics. The review procedures and materials for orienting reviewers must be described in detail within the proposal. The Bias Review Committee will review all of the previously stated items to ensure that they are fair and free of bias for all students, including those who are visually and/or hearing impaired, or deaf.


The Bias Review Committee will meet once prior to online field-testing, and may meet at least once more prior to final test development.
The vendor will provide a minimum of one qualified item writer to facilitate the Bias Review Meetings. The Agency may conduct a separate Bias/Sensitivity Review, with the results being provided to the vendor. Agency personnel will be in attendance during the review sessions. All proposals must provide examples of vendor protocols and procedures for committee review.
3.2.2.5.4.3 Face-to-Face Review Meetings

Following the Algebra I EOC online field test, the vendor will provide a plan for Face-to-Face Item Review meetings with the Agency staff at the vendor’s site, or at a different location that is mutually acceptable for all participants. The vendor must have a minimum of one item writer with Algebra I expertise available to successfully conduct the Face-to-Face Review meetings. Meeting plans must include activities, timelines, and the number of participants required for conducting these review meetings.


At least two weeks prior to the Face-to-Face Review, the vendor will provide the Agency with a written report including revisions of the test items, and comments and suggestions on the content and editorial issues provided by the Content Review and Bias Review Committees. All proposals must provide examples of vendor protocols and procedures for committee review.
3.2.2.5.5 Form Development Process

3.2.2.5.5.1 Field Test Development

The vendor will be responsible for developing and producing field/pilot test forms, manuals, and ancillary materials that resemble, to the extent possible, the proposed online operational assessment. All proposals must provide examples of vendor protocols and procedures for committee review.


The vendor will design the field/pilot test to provide information to evaluate test items and to evaluate the overall test design. The vendor will describe the characteristics of the field test providing calibration results, scoring procedures, and other data from the 2007-2008 field-test. The documentation should also address the reliability of scores and dimensionality of the proposed operational tests, and include multiple methods for estimating the underlying dimensionality of test data as determined in consultation with and approved by the Agency. Written technical data of the results of this investigation will be presented to the Agency before October 2008. The vendor will make recommendations for scaling approaches in the event of appreciable multidimensionality. The contractor will communicate frequently with the Agency during the field/pilot testing period.
It is the responsibility of the vendor to group the newly developed items into item sets to be included in the field/pilot test forms. Using items that have been revised, as determined after the item review meetings, the vendor in conjunction with the Agency will develop a plan that indicates which of the items should be field/pilot-tested. Agency will review the vendor’s plan and approve, or request revisions. The determination of whether, or not, each item goes into the item pool for use in operational tests is to be a joint decision between the vendor and the Agency with the Agency having final approval. The Agency is committed to reviewing products submitted by the vendor as efficiently as possible. At a minimum, the vendor must collect information on CSO alignment, p-values, point biserials, and dif analysis.
3.2.2.5.5.2 Operational Form Development

The requirements for constructing online operational tests include creating test construction specifications for building the forms. Vendors must detail their specification and form building processes. In addition, test construction must be supported by sophisticated computer software that will generate test characteristics.


Two operational online test forms measuring the 21st Century WV CSOs (along with ancillaries) will be selected and moved through the vendor’s form development process within a reasonable timeframe, agreed upon by the vendor and the Agency, following the online field/pilot test. The vendor will design and produce any online graphics needed for the test forms.
The Algebra I EOC online assessment pilot/field test must be completed by a minimum of 5,000 students that are representative of students in grades 7-12 presently enrolled in Algebra I. The vendor in conjunction with the Agency will identify a representative sample of schools to participate in the pilot and the vendor shall provide all necessary materials and technology (software, hardware, connectivity) to complete the pilot assessment.
The vendor will develop an online platform that allows for all data to be collected and aggregated at the state, county, school, grade, class/teacher and student level using the student WVEIS number for identification of the results. The vendor shall provide evidence of the validity of online testing and scoring in the context of a large-scale assessment.
The Agency elects to participate in the Algebra I EOC operational forms development. The Agency may choose to modify the design of the test forms and ancillaries prior to any test administration, within the constraints of the specifications for test forms and ancillaries provided. The Agency reserves the right to change this configuration, if necessary, through change orders, or appropriate contract amendments. Throughout the forms development process, the Agency is committed to reviewing products submitted by the vendor as efficiently as possible.
The vendor must make available to the Agency an online review prior to the actual site activation. Agency has final approval of all test forms prior to site activation/administration.
3.2.2.5.6 Ancillary Product Development

The vendor must develop and make available ancillary products/materials for the Algebra I EOC Exam including, but not limited to:



  • Test examiner’s manual

  • Test coordinator’s manual

  • Mathematics reference sheet

These ancillary materials must be available from the Agency website for the Algebra I EOC testing for Algebra I teachers and County Test Coordinators. The ancillary products/materials must be printable from the website. The Agency will have unlimited access to these materials twenty-four hours per day, seven days per week for the life of the contract.


In addition, the vendor will develop for Agency approval and print all publications, materials, and/or forms, as needed, in compliance with Agency printing specifications. All print materials proposed by the vendor must be approved by the Agency. When printable forms and ancillaries are available for teachers, they should print with minimal white space between items. In addition the vendor will print any additional materials needed in the implementation of this project.
3.2.2.5.6.1 Test Examiner’s Manual

The vendor will develop an online Test Examiner’s Manual, which will contain instructions for administration of the Algebra I EOC Exam, general information about how to conduct the assessment and specific test instructions. The manual will also contain information about the security of testing materials. The manual will be available to Algebra I teachers in electronic format on the Agency website prior to the start of the EOC testing. All test manual materials must be printable from the secure website during that time period.


The vendor will submit to the Agency, within an agreed upon time frame, the proofs of the Test Examiner’s Manual to ensure that the testing instructions are consistent with the testing instruments. The vendor will ensure that the manual proofs, and final electronic copy, are free of typographical and format errors before they are submitted to the Agency, or placed online. The vendor will provide a sample of the printed manual and/or online instructions for committee review.
3.2.2.5.6.2 County Test Coordinator’s Manual

The vendor will develop an online County Test Coordinator’s Manual, which will contain coordinator-level instructions for handling and administering the Algebra I EOC Exam. The manual will also contain information about the security of testing materials. The manual will be available to county test coordinators in electronic format on the Agency website period prior to the start of the EOC testing. All test manual materials must be printable from the website during that time period.


The vendor will submit to the Agency, within an agreed upon time frame, the proofs of the County Test Coordinator’s Manual to ensure that the testing instructions are consistent with the testing instruments. The vendor will provide a sample of the printed manual and/or online instructions. The vendor will ensure that the manual proofs, and final electronic copy, are free of typographical and format errors before they are submitted to the Agency, or placed online. The vendor will provide a sample of the printed manual and/or online instructions for committee review.
3.2.2.5.7 Art and Production

While the Algebra I EOC exam is to be developed and delivered in an online format, the design should be interesting and attractive. Design includes the organization, format, color selection, font style and size, screen captures that provides evidence that the screen resolution does not compromise test results, reports of assessment results, information publications, and other online, or printed materials. The vendor will produce all graphics, charts, and illustrations for the products for which it is responsible. All graphics, charts, and illustrations will be age appropriate and approved by Agency. The vendor must submit examples of online test products created for other programs as part of the proposal.


3.2.2.5.8 Accommodations

The online Algebra I EOC system must be accessible to visually impaired students. The vendor must provide in detail the adjustments that will be made to the online system that will provide equal access to visually impaired students and students with disabilities. The online large print screen will meet, or exceed, the font requirements suggested by the American Printing House for the Blind (minimum of 18-point type).


3.2.2.5.8.1 Braille

Because some students may be required to take the Algebra I EOC exam in a Braille format, the two equated online tests should be suitable for brailling. The vendor should make suggestions on how to provide a secure Braille exam that follows APHB requirements (Refer to http://www.aph.org/) for Braille.


3.2.2.5.8.2 Online Large Print Screens

The online Algebra I EOC exam system must be accessible to visually impaired students for online administration. The vendor will enable the testing materials to be enlarged online to meet, or exceed, the font requirements suggested by the American Printing House for the Blind (minimum of 18-point type) without distorting, or compromising, the testing materials. The vendor is responsible for providing a screen resolution that does not compromise test results. The enlarging of print will not affect the validity and reliability of testing principles, nor technical factors such as test bias, fairness of items for all students, readability of test directions and test items and item analysis. The vendor will provide screen captures showing the enhanced screens.


3.2.2.5.9 Content/Form Management System

3.2.2.5.9.1 Introduction

The vendor shall have the ability to acquire and manage items, statistical data, item codes, test booklet/online formats, and other pertinent information from their own (and subcontractors’) item development, scoring, and psychometric systems. The Agency will have continuous access to these banks on a twenty-four hour, seven days per week basis. The data (the items and related information) will be placed into a secure content management system that the Agency can access using either web-based technology, or other methods proposed by the successful vendor. The proposal must contain a detailed plan to continually acquire and inventory all West Virginia content and item/form data across the life of the contract. At an absolute minimum, it will be the responsibility of the vendor to update and maintain the actual data in the management system after each field test, or operational administration. Item information will be updated each time an item is used either as a field test or operational item. If applicable, transferring existing information between vendors’ management systems will be the responsibility of the primary vendor as designated by the Agency. The plan must also provide details of how the items and related material will be converted to the vendor’s content management system, if applicable. Past experience suggests that transfer of items, their accompanying graphics, and their statistical information from one system to another is problematic and time consuming. Historical information will be retained for all items, including any developed prior to this project. Vendors must build this consideration into their proposals.


The vendor shall provide assurances/solutions in the proposal to ensure this level of availability. The resolution images of items, item graphics, and reading passages in the system must be considered. Vendors must provide adequate quality controls on the accuracy of the system and the data for committee review. Quality control checks of the data in the content management system are mandatory and vendors must specify the procedures to be utilized for this purpose.
The system and the actual content management system must be in a form meeting the requirements of this section and shall maintain the ability to use historical information about all items. Items developed and utilized during the course of the project described herein will be added to the system according to the schedule approved by the Agency, and these items will become the property of West Virginia.

.

3.2.2.5.9.2 Descriptive Information Content/Form Management System

Descriptive information associated with each item to be included in the Content/Form Management System should include:


  • Algebra I content

  • Grade

  • Reporting category

  • Content standard and objective alignment

  • Depth of Knowledge levels

  • Item identification and classification codes

  • Item type

  • Test forms

  • Position in the online forms

  • Number of answer choices

  • Answer key (list of correct responses)

  • Passage/stimulus name

  • Images of items

  • Item graphics and other art or stimuli associated with items

  • Administration date

  • Status (field-test or operational)

  • History of use including test form(s), page, and item number

  • History/tracking of changes/edits made to items during reviews, etc.

  • Identify the person who requested or authorized changes/edits

  • Item alignment


3.2.2.5.9.3 Psychometric Information for Content/Form Management System

Of particular significance is how all of the psychometric data will be collected, analyzed, verified, and stored for future use in building test forms. The content/form management system must be capable of retrieving and utilizing both traditional and classical statistics for use in test item selection and test construction procedures. The plan must include what information will be included, how missing parameters, or statistics, will be acquired, or computed, any transformations needed of parameters using historical data files, and any limiting issues anticipated, along with proposed solutions.


The content management system must be able to incorporate item data that may include when applicable:

  • Maximum item information

  • Difficulty values

  • Classical item analysis for distracters

  • DIF statistics

The vendor may suggest the collection of additional data. If costs are associated with this collection, the vendor must supply costs within the proposal. The vendor will provide data as required by the Agency for the field test as well as for the operational administrations.


The Algebra I EOC online assessment pilot/field test must be completed by a minimum of 5,000 students, representative of students in grades 7-12 presently enrolled in Algebra I. The vendor in conjunction with the Agency will identify a representative sample of schools to participate in the pilot and shall provide all necessary materials and technology (hardware, connectivity) to complete the pilot assessment.

3.2.2.5.9.4 Software/Hardware Concerns for Content/Form Management System

Security of the content/form management system is of utmost importance to the Agency. The Agency must have twenty-four hours, seven days per week continuous access to this secure site. All update costs for the software/hardware must be defined within the proposal and costs must be honored through the life of the contract.


3.2.2.5.9.5 Optional Content/Form Management System Tasks

Vendors may propose additional content/form management system tasks, or activities, if they will substantially improve the results of the project. The additional tasks or activities must be described in detail within this section, yet separated from the required items in the cost proposal.


The vendor must make available the following (not limited to) services online:

  • Tutorial service that will prepare the student to perform procedures required while taking the Algebra I EOC exam in this system

  • A process within the system that allows the student to return to previous items to “check” or “re-do” an item

  • WVEIS unique identifier (username and password) for students to log on to system

  • The vendor must provide quality technical support for all testing programs and any additional products purchased to include online services. The vendor must provide technical support via toll-free telephone, e-mail, or fax to assist teachers with technical/program questions and concerns. Other support technologies may be submitted for committee review. Technical support should be available from 8am – 8pm EST. Vendor responses must be provided within 24 hours of submission. The vendor must include a “HELP” function and tutorial within the online platform for teachers, administrators, and/or students to utilize. The vendor will provide technical support service for all West Virginia users, with preferential treatment for classroom teachers. The vendor must provide examples and references for committee review.

  • A roster of students feature for teachers


3.2.2.5.9.6 Copyright Issues

The item development plan must include a schedule for acquiring copyright permission as needed for new development and uses, as well as for future uses of any existing passages/items. Vendors should fully describe their experience with obtaining and retaining copyrights, the processes used to perform this, and a description of the specific personnel to be assigned to this task and their experience working with copyright materials.


The vendor is liable for assuring all copyright permission acquisitions The vendor’s proposal should explicitly make assurance that the Agency will be held harmless should a vendor fail to acquire copyright permissions.
3.2.2.5.9.7 Examples of Items

The vendor must provide examples and formats of test items. All examples must show thinking skill level and alignment to the 21st Century West Virginia Algebra I Content Standards and Objectives.


3.2.2.5.9.8 Examples of Screen Layouts

The vendor should provide examples of screen captures of the vendor’s choice from the online test site demonstrating screen layout. From the screen captures the online layout must demonstrate the font size, white space, passage layout to item, presentation of answer options, graphics, method of communicating “stop” and “go” options to students, numeration, and submission conformation screens.


An online review prior to site activation must be provided to the Agency. The Agency must sign-off and approve the content, report features, the online environment, and system operation prior to activation/administration of the exam.
3.2.2.5.9.9 Validity

In alignment with the Agency’s requirements for technical excellence, the vendor will be responsible for establishing and documenting various evidences of validity. This will include, but not necessarily be limited to, the following:



  • Evidence of content validity: the alignment to WV CSOs, test blueprints, item specifications, and test items (See Section 3.1.2)

  • Evidence that test item formats measure the intended content

  • Evidence of the interrelationship among standards

  • Evidence items were chosen based on test specifications

  • Evidence that alternate forms cover the same content

Vendors must provide a plan for demonstrating the validity evidences mentioned above, along with any others, based on their experience and emerging statistical theory.



3.2.2.5.9.10 Technical Information/Documentation

The vendor will be responsible for providing documentation of all technical data associated with each field and operational test. The content of the reports shall include validity and reliability information, scaling, and item statistics. This data will provide sufficient information to allow for an independent evaluation of the quality of the assessment. The data should include tabular and graphic displays of data to illustrate the characteristics and quality of test scores.


3.2.2.5.10 Online Assessment Technology

The Agency requires costs for an efficient web-based online assessment system. The online system must produce results that are valid and reliable. The vendor must provide a web-based system that is secure to minimize cheating and testing violations; and protects both the testing environment and student data. Student scores, data, test results and analysis will belong to Agency. With all assessment, the vendor will provide research-based validity and reliability information and technical data. This information will be provided for committee review.


Vendors must provide evidence that their system will work with the existing technology infrastructure of the Agency and the West Virginia county school systems. (Refer to Section 3.1.1) The vendor must insure that all schools using an electronic vendor product are able to successfully utilize the system. The vast majority of schools in West Virginia have a partial T1 or a full T1 for connectivity to the Internet. Fewer than ten schools use a dial-up system for Internet access. All Internet traffic is routed through the statewide infrastructure before connecting to the public Internet.
If a West Virginia school is unable to access the online assessment system, or if a student is banned from access by the school, or a student is denied access by parents, the vendor must provide an alternative method of testing students and uploading their assessment results and/or scores into the online system.
The user and administrative interfaces must be user friendly and easy to navigate and should utilize technology that will operate with minimal bandwidth usage to ensure prompt application response times. The vendor will provide screen captures of the user and administrative interfaces and must provide documentation on bandwidth usage.
The vendor must provide quality technical support for all testing programs and any additional products purchased to include online services. The vendor must provide technical support via toll-free telephone, e-mail, or fax to assist teachers with technical/program questions and concerns. Other support technologies may be submitted for committee review. Technical support should be available from 8am – 8pm EST. Vendor responses must be provided within 24 hours of submission. The vendor must include a “HELP” function and tutorial within the online platform for teachers, administrators, and/or students to utilize. The vendor will provide technical support service for all West Virginia users, with preferential treatment for classroom teachers. The vendor must provide examples and references for committee review.
The online system must include a secure easy-to-manage system of rostering/rerostering students that is efficient and that allows easy transfer of students from group, class, grade, school, or county. The vendor must describe the procedures for rostering/rerostering for committee review.
The testing platform must provide:

  • an online procedural tutorial that teaches students how to use the testing system

  • an easy navigate interface within the exam

  • the ability to easily to return to exam questions when a student has either stopped the exam for a brief or extended time, or when a student may have skipped a question during testing

  • a feature that shows both answered and unanswered exam items

  • a confirmation request feature that students must respond to before exiting the exam

  • clear, easy-to-view graphics that support ease of readability and focus

The vendor must provide for committee review screen captures of the following items: 1) student log-on screen; 2) student test format; and 3) student procedural tutorial demonstrating how to use the testing system.

The vendor shall conduct a pilot of online testing and scoring for the future administration of the Algebra I program. As part of the pilot, the online assessment must be conducted in 15 counties for a minimum of 5000 students. The vendor in conjunction with the Agency will identify a representative sample of schools to participate in the pilot and shall provide all necessary materials and technology (hardware, connectivity) to complete the pilot assessment. The vendor shall provide scoring of assessments completed in the online pilot. These results will not be distributed to students, schools, or counties.
The vendor shall provide evidence of the validity of online testing and scoring in the context of a large-scale assessment. The vendor should provide an example of such a study for committee review.
This system should possess, but not be limited to, the following features:


  • scalability

  • product support

  • classroom reliability

  • user statistics

  • zero downtime for the hosting servers

  • tutorial for professional development

  • handicap accessible accommodations

  • Universal Design Principles

  • accept uploads of student/user data from the West Virginia Educational Information System (WVEIS) in at least an agreed upon CSV formatted file

  • encrypted data storage

Any online system must meet and exceed the W3C Web Content Accessibility Guidelines (www.w3.org) in order to guarantee that all students including students with disabilities can effectively utilize the assessment system.


Any upgrades to the software/platform must be provided at no cost to the Agency for the life of the contract. The vendor may suggest other features, functionality and/or performance enhancements to the software, hardware, or testing platform. These features, functions, or enhancements must be described in detail and costs must be provided for committee review.
3.2.2.5.10.1 Technology Requirements

For General Requirements refer to section 3.1.1. The vendor will assure reliable delivery of all online assessments. The vendor will provide documentation that the platform has built-in features to prevent the loss of student data both during administration and during scoring/reporting. The vendor must provide for committee review, the vendor specifications for prevention of data loss.


The vendor should describe and provide evidence on how the proposed technology solution will work in this environment and identify the following:

  • Minimum hardware requirements for classroom use. (The minimum hardware requirements for your software should not exceed the configuration in Section 3.1.1 Table 2. Your software should operate on the above classroom computer configuration as well as configurations found in the Digital Divide to serve the maximum number of students.)

  • Minimum hardware requirements at other locations.

  • Minimum software requirements including operating system, browser, etc.

  • Configuration of the technical route/specifications from student access/input to final report.

  • Bandwidth requirements.

  • Software specifications for processing input, how data is saved, how often data is sent to storage or remote site, etc.

  • Technical issues with firewalls, desktop security programs, etc.

  • Any other technical requirements or issues.


3.2.2.5.10.2 Security

The vendor must provide evidence that the system is secure. The system will meet federal compliance regulations for both FERPA (Federal Educational Rights and Privacy Act) and COPPA (Children’s Online Privacy Protection Act of 1998). The student level data must be encrypted and located behind a secure firewall.


The Agency requires a system that has a locked-down Desktop that will not allow students to leave and re-enter the program or go to other Internet sites, programs, or files during test administration.
The program must be able to track student users via a secure system of user IDs utilizing each student’s WVEIS number. The testing platform must include a user interface that will be password protected with unique alpha-numeric passwords that are changed on a regular schedule, or for each individual assessment. The vendor will work with Agency in order to obtain a list of users to be loaded into the system and to determine the password and user name schemes to be employed.



Download 5.9 Mb.

Share with your friends:
1   ...   11   12   13   14   15   16   17   18   ...   38




The database is protected by copyright ©sckool.org 2020
send message

    Main page