TOM TORLAKSON, State Superintendent of Public Instruction
California Long-term Assessment Plan.
Summary of Key Issues The California Department of Education’s (CDE) contract with Educational Testing Service (ETS) requires ETS to assist the State Board of Education (SBE) and the CDE in developing a long-term assessment plan.
Background In 2002, the CDE and the SBE published a long-range assessment plan that facilitated the development of assessments currently administered by way of the Standardized Testing and (STAR) Reporting Program. In March 2006, the SBE approved the budget with ETS that included this planning task. ETS has continued to work with the CDE on revisions to the long-term assessment plan.
With California’s adoption of the Common Core State Standards (CCSS) and becoming a governing member of the Smarter Balanced Assessment Consortium, the CDE has been preparing the assessment system for transition. California’s commitment to the CCSS and Smarter Balanced presents challenges as changes are made to curriculum and instruction, and the administration and reporting of new assessments. California’s goal to provide the best and most efficient assessments possible for its teachers and students is reflected in the January 2013 report by the State Superintendent of Public Instruction (SSPI), Recommendations for Transitioning California to a Future Assessment System. To help the department and board address these challenges in the transition of the assessment system, the draft of A Long-Term Assessment Plan for the California Assessment System, presented by ETS, is provided in Attachment 1. The attached draft of the plan has been reviewed by the SBE assessment liaisons and SBE staff and reflects their feedback. This plan is divided into two major sections that identify “what must be done today,” “what might be done tomorrow,” and “what could be done in the future.” The first section identifies immediate tasks that must be accomplished over the next 18 months and intermediate considerations that may be addressed over the next 3 to 5 years. Longer term possibilities to further the purpose of the California assessment system are presented in the plan’s appendix.
The draft of A Long-Term Assessment Plan for the California Assessment System, provided by ETS, will be presented to the SBE at the July 2013 State Board Meeting for discussion.
Attachment(s) Attachment 1: A Long-Term Assessment Plan for the California Assessment System
A Long-Term Assessment Plan
for the California Assessment System
June 24, 2013 - Draft
Educational Testing Service
A report submitted under the direction of the California Department of Education as
Thoughtful Choices: The Future of Assessment in California 9
Immediate Tasks & Intermediate Considerations of the 12 Recommendations 12
Table 1: Immediate Implementation Tasks for Recommendation 1 12
Intermediate Considerations for Recommendation 1: 13
1.1 Transition Checklist 15
1.2 Limited Form Release of Suspended Tests 17
Table 2: Immediate Implementation Tasks for Recommendation 2 19
Intermediate Considerations for Recommendation 2: 20
2.1 Communication Documents to Stakeholders 20
2.2 Technology-Based Assessments in Other Subjects 21
Table 3: Immediate Implementation Tasks for Recommendation 3 23
Intermediate Considerations for Recommendation 3: 23
3.1 California College Ready Indicators with Augmentation 24
3.2 College Ready Indicators and System Migration to the CCSS 25
3.3 College Ready Indicators and Technology-Based Assessments (TBAs) 25
3.4 Validity Evidence of College Ready Indicators 26
Table 4: Immediate Implementation Tasks for Recommendation 4 28
Intermediate Considerations for Recommendation 4: 29
4.1 Release of the Next Generation Science Standards (NGSS) 30
4.2 Comparability of Online versus Paper-Pencil Testing (PPT) 30
Table 5: Immediate Implementation Tasks for Recommendation 5 33
Intermediate Considerations for Recommendation 5: 33
5.1 Availability of the Next Generation Science Standards (NGSS) 34
Table 6: Immediate Implementation Tasks for Recommendation 6 35
Intermediate Considerations for Recommendation 6: 36
6.1 Research and Policy Considerations for Content Assessments in a Language other than English 37
6.2 Psychometric Effect of Sample Size 39
6.3 Professional Development for Teachers of English Learners 39
Table 7: Immediate Implementation Tasks for Recommendation 7 40
Intermediate Considerations for Recommendation 7: 41
7.1 End-of-Course Exams 41
7.2 Exams in Non-ESEA Content Areas 42
7.3 Calendaring of Non-ESEA Content Exams 43
7.4 Sampling of Students and Items in Non-ESEA Content Exams 44
Table 8: Immediate Implementation Tasks for Recommendation 8 46
Intermediate Considerations for Recommendation 8: 48
8.1 Models of Interim Assessment 50
8.2 Models of Formative Assessment 51
8.3 Implementation Considerations of Interim Assessment Components 52
8.4 Implementation Considerations of Formative Assessment Components 53
Table 9: Immediate Implementation Tasks for Recommendation 9 54
Intermediate Considerations for Recommendation 9: 55
9.1 Smarter Balanced as the Next CAHSEE 56
9.2 Optional Voluntary Exams 57
9.3 Successful Course Completion 58
9.4 Future EOC Exams 59
9.5 Matriculation Exams 60
Table 10: Immediate Implementation Tasks for Recommendation 10 60
Intermediate Considerations for Recommendation 10: 61
10.1 Exploration of Matriculation Exam Options 63
Table 11: Immediate Implementation Tasks for Recommendation 11 65
Intermediate Considerations for Recommendation 11: 66
11.1 Comparability via Smarter Balanced Field Testing 67
11.2 Comparability via Smarter Balanced Operational Administration 68
Table 12: Immediate Implementation Tasks for Recommendation 12 69
Intermediate Considerations for Recommendation 12: 70
12.1 Alignment and Instructional Sensitivity 70
12.2 Validity, Utility, and Impact 71
12.3 Scale Stability and Performance Standards 71
Appendix: Long-Term Possibilities 75
A Vision toward the Future 75
Develop additional item types that use student performance to assess more demanding constructs across all content. 80
Use artificial intelligence scoring of constructed responses when appropriately reliable, available, and beneficial. 83
Consider metacognitive factors in determining college and career readiness. 85
Transition to technology-based administration through a considered approach. 87
Reduce the number of students tested when information is used for more global decisions. 90
Strengthen security of administration according to stakes of the exam. 93
Provide real-time results for computer-scored tests. 95
Provide diagnostic information about the next steps in the teaching and learning process. 97
Articulating a coherent assessment system. 99
Articulating a technically defensible process. 100
California, like many other states, has significantly increased its expectations for students: those graduating high school are expected to be ready for college or careers. Like the other 44 states that have adopted the Common Core State Standards (CCSS), California must wrestle with the instructional changes required and how to measure student and school progress.
But, while the challenges are similar, California is unique. The state is bigger, more diverse, and more complex. California has more than 1,000 school districts with 300,000 teachers serving 6.2 million students, a third of whom live below the poverty line and a quarter of whom are English learners. The state has faced multiple years of multi-billion dollar budget cuts, and the schools have borne the brunt of the reductions.
The uniqueness of California is a critical context for the discussion that follows. Only through consideration of California’s particular needs can we determine what kind of assessment system the state might consider going forward and how to transition from what is to what could be.
For the past 14 years, California’s Standardized Testing and Reporting (STAR) program has annually tested students from early elementary through high school. These tests measure student performance against California’s rigorous academic standards, which were established in 1992-1993. In 2010, California adopted the Common Core State Standards, which were developed by a coalition of 48 states. In adopting them, the state committed to the philosophy that a common set of standards provides greater equity for all students (see sidebar).
High standards that are consistent across states provide teachers, parents, and students with a set of clear expectations that are aligned to the expectations in college and careers. The standards promote equity by ensuring all students, no matter where they live, are well prepared with the skills and knowledge necessary to collaborate and compete with their peers in the United States and abroad. Unlike previous state standards, which were unique to every state in the country, the Common Core State Standards enable collaboration between states on a range of tools and policies, including: the development of textbooks, digital media, and other teaching materials aligned to the standards; and the development and implementation of common comprehensive assessment systems to measure student performance annually that will replace existing state testing systems; and changes needed to help support educators and schools in teaching to the new standards. (NGA and CCSSO 2010)
In 2011, California became a governing member of the Smarter Balanced Assessment Consortium, one of two federally funded groups of states developing tests to measure student progress against the new standards. Smarter Balanced expects to produce tests to be used during the 2014-15 school year in mathematics and English language arts. These tests, in grades 3-8 and grade 11, will be what are called ‘computer adaptive’ tests that are electronically administered and that adjust the difficulty of the questions based on the student’s responses.
California currently plans to employ these new tests from Smarter Balanced, and the state’s participation in this consortium sets a framework for constructing a comprehensive assessment system that California will develop. Since the adoption of the CCSS and the subsequent commitment to the Smarter Balanced Assessment Consortium, the California Department of Education (CDE) has been preparing the assessment system for this transition. The CDE has devoted a number of resources to this effort, including establishing a Statewide Assessment Transition Office, tasked with overseeing the transition of the assessments to this next-generation system.
In addition, in January of 2013, State Superintendent of Public Instruction (SSPI) Tom Torlakson released Recommendations for Transitioning California to a Future Assessment System (“Recommendations”). This set of recommendations articulates the purposes of the California assessment system and provides guiding principles that should govern the design of the new system. A seminal recommendation is that California continues its commitment to participate in the Smarter Balanced Assessment Consortium (“Smarter Balanced” or “the Consortium” or “SBAC”).
With the expectation that California will participate in Smarter Balanced beginning in 2014-2015, the state has two equally demanding sets of challenges before it. First, over the next 18 months, school and district educators as well as stakeholders within California educational policy must make curricular and instructional changes that support and improve student learning. They must also prepare logistically for the administration and reporting of these new assessments. It is important that the participants in the assessment system and the system itself are ready for these new measures. Second, California recognizes that the Smarter Balanced assessments are only one component of a comprehensive and coherent assessment system. There are several other considerations regarding assessment purposes, contents, and policy that are necessary in crafting a well-rounded next-generation assessment system.
This plan is divided into two major sections that address these two sets of challenges. The first section reviews the twelve recommendations of the State Superintendent of Public Instruction and identifies the tasks over the next 18 months that we suggest the state complete to accomplish a specific recommendation or to position California to accomplish the recommendation shortly thereafter. Where appropriate, we provide additional considerations in the intermediate term related to the recommendation. California would need to review these recommendations in light of its current contract structure and budget allocations: we have not reviewed these recommendations in light of current contract obligations and budget. The second section within the appendix identifies considerations in the longer term designed to further the purpose of the California assessment system. While these intermediate and long-term recommendations may be beyond any current contract, it is prudent for the state to consider them now if they wish to pursue one or more of these considerations in the coming years, as their implementation will often require years of planning, which is true in any large-scale assessment system.
Thus, this report is designed to identify immediate activities, intermediate considerations, and long-term possibilities. In doing so, we provide recommendations for the focus of the state in developing its assessment system: not so focused on the immediate activity that future options are not anticipated, while not so concerned with the far horizon that little attention is given to the immediate path. This plan attempts to identify what must be done today, what might be done tomorrow, and what could be done in the future. The reality, particularly given the state’s fiscal situation, is that policymakers will have to make difficult choices on what assessment system they want and what they can afford.