ROSE MINCE JULY 2010 Assessment is a critical component of the Middle States Commission on Higher Education Characteristics of Excellence. The college-wide launching of the CCBC reaccreditation process has brought renewed emphasis on planning, assessment, and documentation throughout the institution. The Learning Outcomes Assessment Advisory Board (LOAAB) as a whole and many of its individual members have been assigned leadership responsibilities in the reaccreditation process, especially for standards that have a strong assessment component. During the past year, LOAAB and the Office of Instruction, through the office of the Dean of Instruction for Curriculum and Assessment, has continued to support high-impact course-level assessment, General Education assessment, program-level assessment, and institution-level assessment. The report from each of these areas follows this introduction.
During the 2009-2010 academic year, the Learning Outcomes Assessment Advisory Board continued to provide guidance, direction, and planning for continued quality improvement of all of the instructional assessment projects and to broaden the adoption of the college’s assessment model across the entire institution. Faculty members and administrators from across the college met monthly to work on these issues. A new highly successful endeavor that was launched in fall 2009 was a series of faculty workshops that were jointly sponsored by LOAAB, the Center for Excellence in Teaching and Learning, and the General Education Review Board. These “Education Stimulus Package” workshops were facilitated by faculty from each of the academic schools and provided a venue for faculty to share practical, hands-on strategies to engage students. The workshops will be repeated in 2010-2011, with all seven academic schools participating in one set of workshops in the fall semester.
The primary area of focus for LOAAB in 2010-2011 will be online teaching and learning. In spring 2011, LOAAB will resurrect Assessment Appreciation Day, an annual event that was held for several years when CCBC’s assessment initiatives were being introduced, piloted, and initially implemented. The purpose of Assessment Appreciation Day is to recognize the excellent work that CCBC faculty have completed in course, program, and General Education assessment and to share intervention strategies that have been particularly successful in enhancing student learning.
The academic deans identified several new high enrollment courses to begin the course-level assessment process. Twenty-one of these projects have been completed, with executive summaries posted on the LOA webpage. An additional four projects were completed in 2009-2010; the full data reports for those projects are included in this 2009 Learning Outcomes Assessment Annual Report. To begin the assessment process, a team of faculty works with the college’s Outcomes Associate to develop a plan for assessing the major objectives of the course. Once the plan is formalized into a Request for Proposal (RFP) and is approved, the project is launched. Faculty guide the project with the support and assistance of the Outcomes Associate and the Dean of Instruction for Curriculum and Assessment. Data collection and analysis is provided by the Office of Planning, Research, and Evaluation. CCBC continues to be one of the very few, if not the only, community college that has the hallmark of external validation as a key feature of this faculty-driven, research-based assessment model. These course-level projects continue to shed light on the most salient factors that support or inhibit learning. The interventions for improvement almost always result in improved student learning and success. When the desired results are not achieved, a plan for continued improvement is developed and implemented.
In the fall 2009 and spring 2010 semesters, General Education courses in scheduled disciplines based on a published three-year assessment cycle collected assessment data using the General Education Assessment Teams (GREAT) process. The GREAT project began with several semesters of pilots prior to full college-wide implementation in 2004. The college has just begun its second full round of data collection. This allows a comparison of the results from the first to the second assessments of the Common Graded Assignments and their accompanying scoring rubrics. All of the GREAT data reports from 2004 to the present are posted on the CCBC Learning Outcomes Assessment Advisory Board/General Education Review Board webpage. The emphasis at this point in the process is to identify and implement the interventions that can be incorporated to enhance student success in the six General Education program skills and to provide faculty with the professional development they need to be successful. When the GREAT project was first initiated, many training workshops were offered for faculty and administrators. To reemphasize the importance of CCBC’s General Education Program for all students and to provide faculty with the professional development that they need, a General Education Symposium, offered in conjunction with the college’s annual Fall Focus, will be offered in August 2010. Ten breakout sessions will be offered as well as lunchtime conversations with content experts. In spring 2010, all full-time faculty members who teach a General Education course were e-mailed an anonymous survey. All of the major topics that were identified by survey respondents will be addressed during the breakout sessions of the symposium.
Program-level assessment is a required component of program review. Seven to ten programs participate in program review each year, and program coordinators submit a plan for assessing their major program outcomes as part of the program review process. Program-level assessment follows the same five stages as course-level projects. The academic deans and program coordinators have done an excellent job during the past few years of reviewing existing courses and program offerings and updating curricula to maintain or exceed the demands of the workplace. During the past three years, a large number of new programs and certificates were submitted to the Maryland Higher Education Commission for approval as new CCBC offerings. CCBC offers the broadest array of certificates and programs for students and has more statewide designated and Health Manpower Shortage programs than any of the other Maryland community colleges. Many out-of-county students come to CCBC to become credentialed in these programs. Continuous review of courses, certificates, and degree programs and submission of needed changes to the Curriculum and Instruction Committee, the College Senate, and, in the case of new certificates and degrees, the Board of Trustees helps ensure that CCBC students are provided the best possible educational offerings.
LEARNING OUTCOMES ASSESSMENT ADVISORY BOARD
ANNUAL REPORT PREPARED BY:
NATASHA MILLER JULY 2010
During the 2009-2010 academic year, the Learning Outcomes Assessment Advisory Board (LOAAB) focused on showcasing the many pedagogy projects that were being conducted in all departments across the college. To showcase these projects two professional development events were provided: one in the Fall with the theme “The Education Stimulus Package” and the second in the Spring entitled “Focus on Firsts.”
The “Education Stimulus Package” workshop was conducted on the Essex campus during the second week of October 2009. Faculty from the following Schools presented workshops on a variety of topics pertaining to best practices in pedagogy: Liberal Arts, Math and Science, and Business, Social Science, Wellness and Education. In Spring 2010 the second workshop was conducted, with the theme “The First…” this event focused on the first experiences (i.e. contact, etc.) students have with faculty and staff. The Schools of Justice, Health Professions, Applied and Information Technology, and Continuing Education and Economic Development provided sessions along this theme. Both professional development workshops were well attended by faculty and staff.
Although these two workshops were the focal point of LOAAB’s academic year, the Board also had the opportunity to discuss an array of other topics ranging from Assessing Student Learning Using VALUE Rubrics for assessing the essential skills of student learning to a review of the processes involved in creating and/or up-dating the college’s Common Course Outlines.
Celeste Stratton and Dionne McMillian Thorne presented the results of the Quality Matters survey that was conducted in 2009 that surveyed faculty on their online teaching practices. The goal of this survey was to identify areas where faculty members were experiencing the most difficulty in teaching online courses and to tailor professional development workshops to meet those needs.
LOAAB members Lynne Mason and Rose Mince discussed their participation in the creation of rubrics that assesses 14 essential skills of student learning. Lynne and Rose served on separate nation-wide teams to create these rubrics. The Board discussed how these rubrics could be utilized at CCBC and it suggested that they could be used to assess capstone courses or in the program evaluation process.
The final meeting of the academic year was spent reviewing some of the Learning Outcomes Assessment projects that were conducted over the past several years. Natasha Miller reviewed the results of Arts 101, Criminal Justice 101, Math 082 and Reading 051. These completed Learning Outcomes Assessment projects showed improvement in mean scores after interventions were implemented. This review allowed the board members to delve into the data and to get a sense of how each discipline implemented their project and the outcomes of these projects.
COURSE LEVEL LEARNING OUTCOMES ASSESSMENT
PREPARED WITH ASSISTANCE FROM THE LOA DATA SUMMARIES PREPARED BY
Final data reports for completed projects, 2009-10
LOA Status Report, June 2010
CCBC continued its established course-level assessment program this academic year, concluding some projects and progressing on others. The course level assessment process utilizes externally validated assessments that directly measure student learning at the course objective level. All assessment projects begin with the development of a Request for Proposal (RFP) and flow through the five stages listed below. Throughout the process, faculty teams attempt to adhere as closely to basic research design as possible. The Planning, Research, and Evaluation Office conducts the data analyses and provides a thorough report for stages 2 and 4. During the academic year 2009-10, Nancy Bogage served as the colleges’ Outcomes Associate and provided assistance to faculty during each of the five stages.
Prior to launching a project, a team of faculty leaders is selected by the academic dean to serve as the primary researchers for each project. An orientation meeting is held to fully brief those leaders about the college’s policies and procedures and to begin framing the outcomes to be measured and the research design to be employed. Faculty then select an assessment instrument, ensure external validation, and outline a timeline for completion of each stage listed below.
Stage 1: Designing and Proposing a Learning Outcomes Assessment (LOA) Project
Stage 2: Implementing the Design and Collecting and Analyzing the Data
Stage 3: Redesigning the Course to Improve Student Learning
Stage 4: Implementing Course Revisions and Reassessing Student Learning
Stage 5: Final Analysis and Reporting Results
Four projects were completed this past year. The completed projects were Arts 104 (Art Appreciation), Philosophy 101 (Introduction to Philosophy), Women’s Studies 101 (Introduction to Women’s Studies), and Criminal Justice 202 (Criminology). The final data analysis reports prepared by Natasha Miller and the executive summaries written by the faculty teams for those projects are included below.
Two new projects will begin in 2010-11, Mathematics 153 (Introduction to Statistical Methods) and Student Development 103 (Career/Life Planning). The proposed LOA for MATH 153, Introduction to Statistics, will collect information on the student’s math background and basic statistical knowledge when entering the class in addition to assessing the level of mastery of the concepts covered in the course. The objectives as listed on the MATH 153 Common Course Outline will be used to guide the instrument for the LOA. Two topics will not be directly assessed to allow the instructor a greater level of flexibility in administration of the instrument. The topics to be excluded are Chi Square Test of Independence and hypothesis testing for two populations.
The SDEV 103 (Career and Life Planning) Learning Outcomes Assessment project will determine the degree to which students are able to prepare a Career Portfolio that includes:
A completed informational interview with reflection paper,
Two summary reports from ONET on two career selections based on major selection and assessment results, and
A written personal reflective statement of life experiences, goals, and accomplishments.
One significantly revised project will be continued, FLSP 101 (Introductory Spanish). Problems with data collection caused unreliable results for the initial data summary. The team will addend the RFP and continue data collection in Fall 2010.
Other projects in various stages are recorded in the LOA Status Report included below.
The Community College of Baltimore County
Office of Planning, Research and Evaluation
August 2008 The ARTS 104 Learning Outcomes Assessment (LOA) project was conducted during the Spring 2008 semester on all three CCBC main campuses. The assessment was distributed to students at the end of the semester, May 2008. The assessment data was combined with other student information such as grades, race/ethnicity and other demographics, to provide a comprehensive description of the students. The first part of this report provides a background of all students enrolled in ARTS 104 including grade distributions and student demographics. The remainder of this summary will focus on 1) the results from the Fall 2007 and Spring 2008 assessments and 2) an analysis of the three objectives set forth in this course. If you have any questions, please contact Natasha Miller, 410-455-4745 or firstname.lastname@example.org.