Avid eoc assessments Advancement Via Individual Determination Grade 11 avid elective Course End-of-Course Assessment Test Item Specifications



Download 0,55 Mb.
Page1/3
Date conversion01.04.2018
Size0,55 Mb.
  1   2   3
AVID

EOC Assessments





Advancement Via Individual Determination
Grade 11 AVID Elective Course
End-of-Course Assessment
Test Item Specifications

Revised: January 2015



Copy Right Statement

The material contained herein may not be published, printed, rewritten, copied, or redistributed, except to make paper copies of the contents only for distribution and use within schools currently contracting with AVID Center and implementing the AVID elective. Copying or transmitting of any kind of this material is strictly forbidden without permission from AVID Center.

Copyright © 2015

AVID Center

San Diego, California

Table of Contents

4 Introduction

Origin and Purpose of the Specifications

Scope of this Document

Overall Considerations


6 Criteria for Grade 11 AVID Elective End-Of-Course Assessment Test Items

Use of Graphics

Item Style and Format

Scope of Test Items

Guidelines for Item Writers
10 Item Difficulty and Cognitive Complexity of Grade 11 AVID Elective End-of- Course Assessment Test Items

Item Difficulty

Cognitive Complexity

Universal Design


17 Review Procedures for Grade 11 AVID Elective End-of-Course Assessment Test Items

Review for Potential Bias and Community Sensitivity

Review of Test Items
18 Guide to the Individual Benchmark Specifications

Benchmark Classification System

Definitions of Benchmark Specifications
21 Individual Benchmark Specifications for Grade 11 AVID Elective End-of- Course Assessment
A–1 Appendix A: Directions for Item Review and Grade 11 AVID Elective End-of- Course Assessment Item Rating Form
B–1 Appendix B: Content Assessed by Grade 11 AVID Elective End-of-Course Assessment
C–1 Appendix C: AVID Item Writer Glossary
D–1 Appendix D: Grade 11 AVID Elective End-of-Course Assessment Test Design Summary
E–1 Appendix E: Course Descriptions for Grade 11 AVID Elective

Introduction
In compliance with End-of-Course Assessments required by State Boards of Education, AVID Center has designed and approved the EOC for the Grade 11 AVID Elective. The AVID Standards are subdivided into benchmarks that identify what a student should know and do following completion of this course. The Grade 11 AVID EOC Assessment measures achievement of students enrolled in this course, by assessing student progress on benchmarks that are assigned to the Grade 11 AVID Elective Course description which are located in Appendix E.

Origin and Purpose of the Specifications

AVID Center and an EOC writing committee of experienced AVID educators developed and approved the Specifications. The Specifications is a resource that defines the content and format of the test and test items for item writers and reviewers. The Specifications indicates the alignment of test items with the Common Core State Standards. It also serves to provide all stakeholders with information about the scope and function of the end-of-course assessment.


Scope of this Document

The Specifications for Grade 11 AVID Elective EOC Assessment provides general guidelines for the development of all test items used in this assessment.


The Overall Considerations section in this Introduction provides an explanation of the AVID concepts and elements assessed by the test. The Criteria for Grade 11 AVID Elective EOC Assessment Test Items section explains the general guidelines for selection and development of multiple-choice items. The Item Difficulty and Cognitive Complexity of the Grade 11 AVID Elective EOC Assessment Test Items address cognitive complexity levels as well as item difficulty and universal design. The Review Procedures for the Grade 11 AVID Elective EOC Assessment Test Items section covers the review processes used to ensure the quality of the stimuli and test items. The Individual Benchmark Specifications section contains specific information about each benchmark. This section provides benchmark clarification statements, content limits, stimulus attributes, content focus, and a sample item for each benchmark.

Overall Considerations

This section of the Specifications describes the guidelines that apply to all test items developed for the Grade 11 AVID Elective EOC Assessment.

Overall considerations are broad item-development issues that should be addressed during the development of test items. Other sections of Criteria for Grade 11 AVID Elective End-of-Course Assessment Items relate more specifically to one aspect of the development (e.g., content limits or stimulus attributes).
1. Most test items should be written to measure primarily one benchmark; however, other benchmarks may also be reflected in the item context.
2. Some benchmarks are combined for assessment; the individual specification indicates which benchmarks are combined.
3. Test items should be appropriate for students in terms of course content experience and difficulty, cognitive development, and reading level.
4. Test items should be written to match a variety of cognitive levels. Each benchmark should be assessed by items of varying cognitive complexity.
5. The reading level of the test items should be grade 11, except for specifically assessed AVID terms or concepts.
6. Test items should assess the application of the concept rather than the memorization of AVID facts unless otherwise noted in the individual benchmark specifications.
7. Some test items may require the student to define and/or apply terms, while other test items will require students to understand terms used in specific context.
8. Test items will not require the student to create a chart, table, or graph.
9. Each test item should be written clearly and unambiguously to elicit the desired response.
10. Test items should not be designed to create disadvantage or exhibit disrespect to anyone in regard to age, gender, race, ethnicity, language, religion, socioeconomic status, disability, occupation, or geographic region.
11. The item context should be designed to interest high school students.
12. All test items should be placed in a real-world context unless otherwise noted in the individual benchmark specifications. The context should lead the student cognitively to the question. Every effort should be made to keep the test items as concise as possible without losing cognitive flow or missing the overall idea or concept.
13. The item content should be timely but not likely to become dated too quickly.
Criteria for the AVID Elective End-of-Course Assessment Test Items
All AVID EOC Assessment test items are in multiple-choice (MC) format. The general specifications on pages 6 through 9 cover the following criteria for the Grade 11 EOC Assessment:


  • Use of Graphics

  • Item Style and Format

  • Scope of Test Items

  • Guidelines for Item Writers


Use of Graphics

Graphics are used to provide both necessary and supplemental information. That is, most graphics contain information that is necessary for answering the question, while other graphics illustrate or support the context of the question. Items may include text, diagrams, illustrations, charts, or tables, unless otherwise noted in the Individual Benchmark Specifications section.


Item Style and Format

This section presents stylistic guidelines and formatting directions that should be followed while developing test items.


General Guidelines

1. The AVID EOC Assessment may be used as a computer-based assessment or as a paper/pencil assessment.


2. Test items should be clear and concise, and they should use vocabulary and sentence structure appropriate for grade 11.
3. AVID concepts should be appropriate to the content covered in the AVID course description. The course description can be found in Appendix E.
4. Test items should have only one correct answer. The words most likely or best

should be used sparingly.


5. The final sentence of all item stems must be expressed as a question.
6. Graphics in test items should be clearly labeled and contain all information

necessary for a student with benchmark mastery to answer the test item correctly.


7. Test item questions using the word not should emphasize the word not using all

uppercase letters (e.g., “Which of the following is NOT an example of . . .”). The word not should be used sparingly.



Multiple-Choice (MC) Items

1. MC items should take approximately one minute per item to answer.


2. MC items are worth one point each.
3. MC items should have four answer options (A, B, C, D).
4. During item development and review, the correct response should be indicated with an asterisk next to the answer option letter.
5. During item development and review, the rationale for distractors (incorrect answer options) should be indicated and placed in its own section on the Item Development Template.
6. In most cases, answer options should be arranged vertically beneath the item stem.
7. If the answer options for an item are strictly numerical, they should be arranged in ascending or descending order, with the place values of digits aligned. When the item requires the identification of a choice from the item stem, table, chart, or illustration, the options should be arranged as they are presented in the item stem.
8. Answer options should be arranged by the logic presented in the test item, by alphabetical order, or by length. Options may also be ordered in reverse alphabetical order or from longest to shortest. Options that are one word in length should be in alphabetical or reverse alphabetical order.
9. Each distractor should be a plausible answer for someone who does not know the correct answer.
10. Whenever possible, distractors should include common AVID Elective misconceptions or

represent conceptual or factual errors commonly made by students who have not mastered the assessed concepts.


11. All distractors should be written in a style appropriate to the question asked. For example, a how question should have distractors that explain how.
12. Paired comparison structure of options should be avoided.
13. Options should have parallel structure whenever possible. Test items should not have an outlier (e.g., an option that is significantly longer than or different from the other options).
14. Test items should not be clued or answered by information in the stem or other options.
15. Options such as none of the above, all of the above, not here, not enough information, or cannot be determined should not be used as distractor rationales.
16. If an option is a single word or a phrase, the option should start with a lowercase letter. If an option is a sentence, the sentence should be conventionally capitalized and punctuated. Options that are imperatives should be treated as sentences.

Context-Dependent (CD) Item Sets

1. The stimulus for the CD set may be an example from the AVID Curriculum or Web-site. The stimulus may include an AVID document, short passage, poem, chart, graph, student writing sample or scenario.


2. The reading level of the stimulus, excluding AVID terms, should be Grade 11.
3. Test items will be written so that students with benchmark mastery use AVID content

knowledge and the information in the passage/stimulus to answer the test items in the set.


4. Test items will not be clued or answered by information in the passage/stimulus or other items in the set.
5. Test items may require the student to analyze, interpret, evaluate, and/or draw

inferences from the information in the stimulus.


6. As many test items as possible should be written to the stimulus. Those test items

should represent an appropriate variety of benchmarks. On a test, a minimum of

two different benchmarks should be assessed in a CD set.
7. CD sets may be titled; however, titles are not required.

Scope of Test Items

The scope of Grade 11 AVID EOC Assessment test items is presented in Appendix B. The benchmarks serve as the objectives to which the test items are written. There may be additional guidelines or restrictions located in the individual benchmark specifications.


This document also contains remarks and examples under various benchmarks giving specific content that potentially could be assessed. The remarks can also be used to focus test items on specific subjects covered by the benchmark, in addition to, but not exclusive of, the list of Content Focus topics found in the Individual Benchmark Specifications section.

Guidelines for Item Writers
AVID Elective item writers must have a comprehensive knowledge of the assessed AVID curriculum and a strong understanding of the concepts. Item writers should know and consistently apply the guidelines established in the Specifications as well as contribute to the goal of developing test content that allows students to perform at their best. Item writers are also expected to use their best judgment in writing items that measure the AVID benchmarks without introducing extraneous elements that reflect bias for or against a group of students.
Item writers for the Grade 11 AVID EOC Assessment must submit items in a particular format and must include the following information about each item. Because items are rated by committees of AVID educators following submission to the state DOE, familiarity with the directions for rating items (found in Appendix A) will prove useful to all item writers.
Format Item writers must submit test items in the agreed-upon template. All

appropriate sections of the template should be completed before the items

are submitted.
Sources Item writers are expected to provide sources for all verifiable information

included in the test item. Acceptable sources include AVID Curriculum, AVID Web Site File-Sharing Documents, Modules on Demand, AVID Online E-learning Courses, and/or public domain texts from Authorama.com. and the Library of Congress.



Correct

Response Item writers must supply the correct response. Each distractor should be a

believable answer for someone who does not know the correct answer.

Rationales must include explanations for these errors.

Submission

of Items When submitting items, item writers must balance several factors. Item

submissions should:



      • include items of varying difficulty;

      • include items of varying cognitive complexity;

      • include items from varying content foci;

      • include the content source(s) for the item;

      • have a balance in location of the correct answer within benchmarks; and

      • be accurate of the content.


Item Difficulty and Cognitive Complexity of Grade 11 AVID Elective

End-of-Course Assessment Test Items

Educational standards and assessments are aligned based on the category of content covered and also on the complexity of knowledge required. The Grade 11 AVID Elective EOC Assessment items, must also reflect these goals and standards. It is important to develop items that elicit student responses that demonstrate the complexity of knowledge and skills required to meet these objectives. The degree of challenge of test items is currently categorized in two ways: item difficulty and cognitive complexity.


Item Difficulty

The difficulty of test items is initially estimated by committees of educators participating in Item Content Review meetings each year. As each test item is reviewed, committee members make a prediction of difficulty based upon their knowledge of student performance at the given grade level. The classification scheme used for this prediction of item difficulty is based on the following:


Easy More than 70 percent of the students are likely to respond correctly.
Average Between 40 percent and 70 percent of the students are likely to respond correctly.
Challenging Fewer than 40 percent of the students are likely to respond correctly.
After an item appears on a test, item difficulty refers to the actual percentage of students who chose the correct answer.

Cognitive Complexity

Cognitive complexity refers to the cognitive demand associated with an item. The cognitive classification system implemented for the AVID EOC is based upon Dr. Norman L. Webb’s Depth of Knowledge (DOK) levels.1 The rationale for classifying a test item by its DOK level of complexity focuses on the expectations made of the test item, not on the ability of the student. When classifying a test item’s demands on thinking (i.e., what the test item requires the student to recall, understand, analyze, and do), it is assumed that the student is familiar with the basic concepts of the task. Test items are chosen for the AVID EOC assessments based on their grade-level appropriateness, but the complexity of the test items remains independent of the particular curriculum a student has experienced. On any given assessment, the cognitive complexity of a multiple-choice item may be affected by the distractors. The cognitive complexity of a test item depends on the grade level of the assessment; a test item that has a high level of cognitive complexity at one grade may not be as complex at a higher grade.


The categories—low complexity, moderate complexity, and high complexity—form an ordered description of the demands an item may make on a student. For example, low-complexity test items may require a student to solve a one-step problem. Moderate-complexity test items may require multiple steps. High-complexity test items may require a student to analyze and synthesize information. The distinctions made in item complexity ensure that items will assess the depth of student knowledge at each benchmark. The intent of the item writer weighs heavily in determining the complexity of a test item. At the end of this section, three high school AVID items illustrate how a single concept may be assessed by test items with increasing cognitive complexity.
The pages that follow illustrate some of the varying demands that test items might make at each complexity level for the AVID EOC Assessment. Note that test items may fit one or more descriptions. In most instances, these test items are classified in the highest level of complexity demanded by the test item. Caution must be used in referring to the table of descriptors that is provided for each cognitive complexity level. This table is provided for ease of reference, but the ultimate determination of item complexity should be made considering the overall cognitive demand placed on a student. Another table provides the breakdown of the percentage of points by cognitive complexity level.

1

Webb, Norman L. and others. “Web Alignment Tool” 24 July 2005. Wisconsin Center of Educational Research. University of Wisconsin-Madison. 2 Feb. 2006. http://www.wcer.wisc.edu/WAT/index.aspx.




Low- Complexity

AVID low complexity test items rely heavily on recall and recognition of previously learned concepts and principles. Low-complexity test items involve the recognition or recall of information such as fact, definition, term, or simple procedure. These items can involve recognizing information and identifying characteristics.


Below is an example of a low-complexity test item that is based on Benchmark AV.11.COLL.1.1. For more information about this item, see page 43.

When first forming a study group, it is recommended that the group members create group norms. What does the term group norms mean?
A. a collection of study skills to use during the study session
B. a list of each member’s grade point average and SAT scores
*C. a set of rules or guidelines that will help the members to work collaboratively and productively
D. a record of the results from a learning styles survey that each member takes before joining the group




Source: (2005) Strategies for Success, (2012) AVID Tutorial Guide, AVID Center. San Diego, CA: AVID Press.
Moderate-Complexity

AVID moderate-complexity test items involve more flexible thinking than low-complexity test items require. Moderate-complexity test items involve the engagement of some mental processing beyond recalling or reproducing a response. They also involve examining relationships, determining cause and effect and determining significance.


Below is an example of the moderate-complexity test item that is based on Benchmark AV.11.COLL.1.1. For more information about this item, see page 43.


Your pre-calculus teacher suggested that the members of your class should form independent study groups to prepare for the mid-term exam. What do you think your group should do at the beginning of their first meeting?

A. Go over the vocabulary and review the summaries of the chapters being tested.


B. Share notes, discuss concerns from previous tests, and brainstorm a list of topics that will be tested.
C. Make a list of difficult items to review, work out a difficult problem together, and write reflections.
*D. Set group norms and expectations, and create a calendar of meetings and topics to cover.




Source: (2005) Strategies for Success, (2012) AVID Tutorial Guide, AVID Center. San Diego, CA: AVID Press.

High- Complexity

AVID high-complexity test items make heavy demands on student thinking. Students must engage in more abstract reasoning, planning, analysis, judgment, and creative thought. These test items require that the student think in an abstract and sophisticated way, often involving multiple steps.


Below is an example of a high-complexity test item that is based on Benchmark AV.11.COLL.1.1. For more information about this item, see page 43.



You are a member of a study group working on preparing for a final exam for Advanced English. Lately the group members have not been getting along and nothing productive is happening at the meetings. What AVID skill can you apply to help the group work more cooperatively?

A. Have a Philosophical Chairs debate on the groups’ problems.


B. Invite the group to revise their Cornell notes from class.
*C. Review group norms and expectations; collaboratively decide on a

plan of action.


D. Teach everyone how to mark the text to support the students who are having reading difficulty.




Productive stud

Source: (2005) Strategies for Success, (2012) AVID Tutorial Guide, AVID Center. San Diego, CA: AVID Press.

The following table is provided for ease of reference; however, caution must be used in referring to this table of descriptors for each cognitive complexity level. The ultimate determination of an item’s cognitive complexity should be made considering the intent of the overall cognitive demand placed on a student. In this table the term AVID methodologies refers to the best practices of implementing and applying WICOR strategies, as well as addressing college readiness skills. In this table the term AVID goals, practices, and procedures refers to all of the Domains of the AVID Standards and meeting the AVID Essentials 5 - 8.


Examples of AVID Activities across Cognitive Complexity Levels


Low Complexity

Moderate Complexity

High Complexity

Identify, recall, or recognize AVID methodology terms.

Apply or infer the effects of AVID methodologies.

Solve or predict outcomes of a problem.

Identify, recall, or recognize AVID goals, practices, and/or procedures.

Identify outcomes of the implementation of AVID methodologies.

Predict a long term result, outcome, or change.

Identify strategies that support the learning process.

Recognize similarities and differences between AVID methodologies.

Analyze similarities and differences of student learning and/or AVID methodologies.

Identify characteristics of AVID methodology practices, and/or procedures.

Explain AVID goals, practices, and/or procedures.

Justify events, actions, or issues regarding AVID goals, practices, and/or procedures.

Use a chart, table, diagram or image to recall or recognize information.

Identify the significance of AVID goals, practices, and/or procedures.

Justify the use and purpose of AVID methodologies.

Use text or documents to recall or recognize information.

Analyze patterns of behaviors or problems.

Assess and synthesize AVID methodologies.







Apply AVID methodologies, goals, practices, and/or procedures to real-life scenarios.







Determine the relationship of AVID methodologies to student success.

The table below shows the target range for the percentage of points by cognitive complexity level on the Grade 11 AVID Elective EOC Assessment.


Percentage of Points by Cognitive Complexity Level for the Grade 11 AVID Elective EOC


Grade

Low

Moderate

High

11 AVID EOC

25% - 35%

55% - 65%

10% - 20%
  1   2   3


The database is protected by copyright ©sckool.org 2016
send message

    Main page