Research-Based Program Development, Implementation, and Evaluation Holly Gritsch de Cordova CharisHerzon Rebecca Anderson University of California Santa Cruz The purposeofthis paper is to discuss the educational benefits ofusing institutionally specific research to understand student academic achievement patterns so as to develop and implement effective academic support services. We will describe several research studies and the academic support programs that we developed based on our findings so as to exemplify our success with this technique at the Universityof California Santa Cruz. On our campus, we focus on developing intensive academic support programs to address two major academic issues: ensuring student retention by minimizing student academic failure and improving student access to “educational equity” and “inclusion in excellence.”
Although almost all of our students meet theacademic standards of UC eligibility (only approximately 2 to 4% per year being offered Admit by Exception status), our research indicates that students who have attendedlow-performing high schools, come fromlow- income backgrounds, and are the first in their family to attend a university, are more likely to experience academic difficulty and fail-out of the university. Additionally, they are also more likely to achieve cumulative Grade Point Averages below those of their peers frommore privileged backgrounds throughout their entire four to six years at UCSC. Therefore, it has become a major goal of Learning Support Services at UCSC to conduct research to identify key academic problem areas and to design course-specific academic support programs.
We will present several examplesto illustrate our strategies including: a study of our two levels of Math below calculus, College Algebra (Math 2) andPre-Calculus (Math 3), that resulted in changes to the configuration of course sections and the implementation of a Math 2 Stretch course, allowing students two quarters to master College Algebra; a study of students academic success in our requiredfreshman composition courses resultingin changes in our placement process and our tutorial support; a study of students’ academic performance in a writing-intensive Latin American/Latino Studies course and the writing- intensive support model that emerged; and research validating our conjecture that at-risk students benefit fromsupplemental instruction and tutoring that resulted in required Academic Success Plans for certain groups ofstudents. In all ofthese instances, we sensed the existence of a serious educational problemthat disadvantaged specific
students. Using our findings, we designed and implemented academic support models, and continually collect quantitative and qualitative evaluation data to monitor our effectiveness in assistingstudents’ toimprove their academic performance.
Academic Support Services as a Response to Educational Inequity Through a longitudinal study of the cohort ofstudents who entered UCSC as frosh in
2005, Learning Support Services confirmed our suppositions that UCSC has not yet succeeded in establishing educational equity for its students. Students fromlow- performing high schools, the first in their families to attend a university, who are often students of color, and are given EOP status,are not achievingeducational excellence as frequently as are their more privileged peers. Figure 1 presents a comparison of cumulative GPA’s of fourth-year EOP and non-EOP students in the 2005 cohort by Academic Division based on their academic majors.
Figure 1 Fall 2005 Frosh Cohort: Percentage of Students Whose Cumulative GPA>2.99 by Current Major Division asof Fall 2008, EOP v. Non-EOP
Arts Engineering Humanities Physical & Social Sciences
It is obvious that EOP students are consistently achieving fewer cumulative GPA’s of 3.0 and above, thus making it less likely that theywill have options to attend graduate school programs that will prepare themfor interesting,well-payingprofessional careers. This was especially true in the Social Science Division, a division that traditionally attracts many EOP students who attend UCSC.
Developing and implementing effective academic support programs to equalize students’ opportunities to achieve academic excellenceat UCSC is a major goal of Learning Support Services. These academic support programs first become available during
students’ first year and continue even through such upper-division courses as senior seminars.
Increasing Student Success in Math 2, College Algebra and Math 3, Pre-Calculus Several years ago it became very clear toLearning Support Services, the Mathematics Department, and the Physical and BiologicalSciences Division that students who enter UCSC underprepared in mathematics wereseriously hampered frompursuing their initially preferred major of study. At UCSC, mathematical competence at the level of eligibility for calculusis required of all majors in theSchool of Engineering, Division of PhysicalandBiologicalSciences, Economics, and Psychology. However, the overall pass rate in Math 2 (College Algebra) wasbelow 75% and the overall pass rate in Math 3 (Pre-Calculus) fluctuated from64 to 97%depending on the quarter and teacher effect. Obviously, we knew that we needed to design extensive academic support programs for these students. Simply supporting these classes with supplemental instruction and tutoring that students were encouraged to use voluntarily was not adequately addressing the students’ learning needs.
In our initial study, we traced the academic achievement trends of students based on their demonstrated preparation for Math 2as assessed by the UCSC Mathematics Placement Examination (MPE) (required of UCSC students prior to enrollment in their first math class). Our first academic intervention was torestructure the required sections for Math
2. Rather than continuing to offer the traditional one hour per weekrequired sections of
25 to 30 students, we offered twice-a-week sections of from12 to 15 students led by trained undergraduate Learning Assistants to all students who scored below the mid-point of an Algebra Readiness Exam. We introducedthis exam, now given on the first day of class, to provide a course-specific assessmentfocus that provides more detailed analysis of students’ preparation for Math 2 than the MPE. These measures improved the pass rates in Math 2 to 82 to 87% (winter, 2005-fall, 2008).
Even while the overall class pass rate was increasing, it became apparent that students who scored below the midpoint of the MPE range for Math 2 continued to struggle. Figure 2 illustrates the differences in Math 2 pass rates for EOP and non-EOP students.
Figure 2 Math 2 Pass Rate vs. MPE for Non-EOP and EOP Students, Fall 2004-Fall 2009
In spite of the improved Math 2 pass rates based on the introduction ofthetwice-a-week sections, many students continued to fail the class. As the sole purpose of Math 2 from a student’s perspective is to prepare him/her for Math 3 or otherMath and statistics classes, we remained concerned about a group of students who, even with the twice-a-week sections, continued to earn non-passing grades in Math 2. One group of students, those who scored low on both the MPE and the Algebra Readiness Exam, seemed to need a different instructional option to master their college algebra skills in order to successfully pass Math 2 and move into pre-calculus or decide
to seek a major devoid of mathematics requirements without the consequences of an F on their transcripts. Therefore, the Math Department, Learning Support Services and the Office of the Vice Provost and Dean of Undergraduate Education developed Math 2
Stretch, a two-quarter, 7 unit Math2 instructional model. In this stretch course students begin and end in the Math 2 lecture setting, but spend the end of the first quarter and the beginning ofthe second quarter in a small section taught by a graduate student Teaching Assistant where they review the first part of the coursematerial and pre-learn the second part ofthe course material. As Table1 will indicate, the pass rate in Math2 Stretch in fall/winter 2010-11 was higher than the pass rate in Math 2 for like students with MPE and Algebra Readiness Examscores in the low ranges.
Table 1 Pass Rates for Math 2 and Math 2Stretch Students with Low MPE Scores and
Algebra Readiness ExamScores
Math 2 Fall 2010
Math 2 Winter 2011
Math 2 Stretch
Having explained our progress in increasingthe likelihood of students’ academic success in Math 2, we will discuss the equal challenge of assisting more students to succeed in Math 3, Pre-Calculus.
A trend that is disturbing isthat students in Math 2 who moveon to Math 3 tend to earn one grade lower in Math 3 thanthey did in Math 2.Table2 illustrates these distressing academic achievement patterns.
Table 2 Math 3 Pass Rates Based on Math 2 Grade and Section Utilization
Once-a- week Math 3 sections
Total Students N
Twice-a- week Math
Total Students N
The discrepancy in passrates based on teacher-effect makes Math 3 data more difficult to study longitudinally. Yet, although student pass rates in Math 3 varied widely, a group of EOP students who scored below the mid-pointof the MPE placement range for Math 3 placement exhibited academic difficulty each quarter similar to that experienced by the same level students in Math 2. Therefore, LSS implemented small twice-a-week sections in Math 3 as an option for students. Based on students’ pass rates, as Table 3 illustrates, students’ have demonstrated improvement when they attend these small, twice-a-week required, Math 3 sections.
Table 3 Math 3 Pass Rates Based on EOP Status, MPE Score, and Section Utilization
% Pass (N)
% Pass (N)
These twice-a-week sections have been helpful toall students but we are still concerned that more programinnovation is needed. We still see the patternwhere EOP students in the MPE score range below the midpoint used for Math 3 class placement continue to struggle to pass Math 3. Twice a week sections are helpful, but we may need to consider a stretch model for Math 3 as well.
It is our continual careful collection and analysis of student achievement data in Math 2 and 3 that has enabled us to propose a series of academic support models that have
proven to be effective. Yet, students entering UCSC with underdeveloped university level mathematical skills continue to be at riskof facing academic difficulty in both required general education and major related classes.