Memorandum date: December 9, 2005 cc: L. Fucaloro, V. N. Okada, K. Russikoff, D. Pan To



Download 42,56 Kb.
Date conversion08.05.2018
Size42,56 Kb.
Memorandum

DATE: December 9, 2005 cc: L. Fucaloro, V.N. Okada,

K. Russikoff, D. Pan


TO: Francis X. Flores, Chair

Academic Senate



FROM: Andrew Moss

A1 Assessment Plan Chair



SUBJECT: A1 Assessment Plan -- REVISED

Please note that the revised GE Area A1 Assessment Plan is attached.


The Area A1 Assessment Committee (L. Fucaloro, A. Moss, V. Okada, K. Russikoff) revised the A1 assessment blueprint in accordance with suggestions made by the Senate Executive Committee on Nov. 30, 2005.
The modified plan now includes a clearer, consolidated outcomes statement (see page 3), as recommended by the Executive Committee.
Thank you for your consideration of the revised plan.

English and Foreign Languages Department

REPORT: GE AREA A1 ASSESSMENT COMMITTEE

SPRING 2005
1. Description of GE Area A1


  • How does GE Area A1 meet the goals of general education as defined in Title 5?

As indicated in the university’s Curriculum Guide, Title 5 of the California Code of Regulations provides that the university shall offer a variety of courses in general education. Among them are courses that will help students “develop the ability to express themselves effectively in both written and oral communication and in critical thinking which includes consideration of common fallacies in reasoning.” Area A1 of the Cal Poly Pomona general education program requires that students achieve proficiency in written communication.




  • What purpose does the Area serve in the university curriculum?

The university curriculum guide also stipulates the following: “Every student seeking a bachelor's degree must take a substantial proportion of coursework designed to develop professional competence. In addition, students must develop the knowledge, skills, and understanding which will enable them to function as intelligent and creative members of a community.”


The ability to write effectively is essential to the development of professional competence and to the enhancement of skills and knowledge required for effective citizenship. Area A1 courses provide students with a foundation to help them write clearly and effectively in their major courses and in subsequent general education courses. This foundation must be reinforced by substantial writing practice and feedback in these other courses.


  • The nature of the GE Area A1 Courses

Students entering Cal Poly Pomona as freshmen meet the Area A1 General Education requirement (Written Communication) by taking English 104 (Freshman English I) or English 103 (College Composition for Speakers of English as a Second Language II). In 2003-2004, 2064 students (550.4 FTES) took English 104, while 246 students (65.6 FTES) took English 103. In addition, a small number of students were exempted from freshman English on the basis of A.P. scores of 3, 4, or 5. The remaining students enrolled at Cal Poly met their requirement by taking equivalent courses at community colleges or other four year postsecondary institutions.


2. Learning Objectives


  • Learning Outcomes for GE Area A1

According to the Academic Senate GE Program Assessment Policy, each GE area committee is mandated to “determine the expected outcome(s)” for its particular area of GE (i.e. for the purposes of assessment). As indicated in the policy report, the committee’s statement “should consist of one or possibly two sentences that reflect the purpose of this area in the university curriculum” (p. 4). To meet this mandate, the Area A1 Assessment Committee (see page 4) selected the following outcome:


Students should be able to write clear, well-organized essays that effectively address purpose, audience, and situation in response to timed-exam prompts. The essays should be free of major grammatical and mechanical errors.
The committee selected this outcome because it addresses writing abilities essential to student performance in a wide variety of classes (i.e. writing clearly and effectively in response to essay examinations and other in-class writing assignments).
3. Evidence Collection and Analysis


  • Methods used to measure student performance related to outcomes

According to the Senate Assessment Policy Report, each area committee is mandated to “identify between two and five pieces of evidence that measure how well the desired outcomes have been achieved. At least one of these pieces of evidence must be from some activity occurring after students have completed the area GE classes” (4). In response to this requirement, the Area A1 Assessment Committee selected the following two pieces of evidence:




  1. A sampling of in-class essays (to be written after mid-quarter in Spring, 2006). The essays, which will be written in response to English Placement Test prompts, will be collected from all English 103 and 104 sections. When evaluated according to a common rubric (EPT Scoring Guide, p. 6), the essays will help provide a program-wide indication of students’ ability to write clearly and effectively in response to timed-exam prompts.

To ensure that the focus is on program assessment, and not on the evaluation of individual students or faculty members, all student and faculty names will be deleted from the essays.




  1. Longitudinal data on student performance (i.e. course grades) in approximately six to eight writing intensive GE courses taken after the completion of English 103 and 104. These GE courses must have a significant writing component that plays a substantial role in the determination of the final course grade (e.g. through essay assignments, research papers, and other writing tasks). The data will consist of grades achieved by a random sample of approximately 150 students who completed English 103 and 104 in the fall of 2001. The data will be controlled for SAT and ACT verbal scores as well as overall GPA.

The longitudinal data, when controlled for test scores and overall GPA, will provide a useful indicator of how students’ preparation in English 103 and 104 correlates with performance in GE courses requiring a significant amount of writing.




  • Plan for collecting and analyzing evidence: a timeline


Fall quarter, 2005 – Winter Quarter, 2006:
Once Senate approval of the Area A1 plan is obtained, the collection of longitudinal data will commence. Dr. Kasimatis has indicated that the Office of Institutional Research, Assessment and Planning (IRAP) will assist the EFL Department in generating a random sample of students who took English 103 and 104 in Fall, 2001. IRAP will also obtain ACT/SAT scores and overall GPA’s for these students.
A list of writing six to eight intensive GE courses will be developed by the Area A1 Assessment Committee. An EFL staff member will then obtain grades achieved in those courses by students in the random sample. IRAP staff will assist the EFL Department in the statistical analysis of the longitudinal data.
Spring Quarter, 2006:
Mid-quarter: Students will write in-class essays. Approximately five essays will be randomly and anonymously selected from all English 103 and 104 sections. The sampling will be followed by a norming and evaluation session similar to those used for Graduation Writing Test readings. The attached rubric will be used in the scoring of the essays.
End-of-quarter: Analysis and reporting of data. The data will be analyzed by the Committee to help determine strengths and weaknesses of the program in assisting students to achieve the indicated outcomes.
4. Dissemination of Results


  • The analysis will be presented by the Committee to all faculty in the EFL Department. Written recommendations for programmatic improvement in 2006-2007 will be discussed by the department faculty.




  • The report (including recommendations) will be forwarded to IRAP and to the Academic Senate.



5. Required Resources


  • As indicated in the Academic Senate GE Program Assessment Policy Statement, the “Office of Undergraduate Studies will arrange for committee support including securing meeting facilities, providing photocopy services, and seeing that calendar schedules are met as well as seeing that all Area Assessment Committee members have the appropriate documents to make decisions” (p. 3).

In addition to these services (e.g., the photocopy service to be used in the evaluation of student essays), the EFL Department will need student assistance (30 hours, or approximately $300) to help EFL staff meet routine staff obligations during the time of data collection. Staff time will need to be dedicated to the collection of student grades for the longitudinal GE study.


6. Area A1 Assessment Committee


  • Committee members include:

Liliane Fucaloro, EFL Department Chair

Andrew Moss, Composition Coordinator, Area A1 Assessment Committee Chair

Victor Okada, Coordinator of Basic Communication Skills

Karen Russikoff, Coordinator of ESL Composition Program
The assessment report was prepared in consultation with Debra Brum, Associate Provost for Undergraduate Studies, and Margaret Kasimatis, Director of the Office of Institutional Research, Assessment and Planning.
7. Additional notes: responses to comments made by Claudia Pinter-Lucke and Margaret Kasimatis:
The following notes are responses to written suggestions made by Claudia Pinter-Lucke and Margaret Kasimatis (Oct., 2005):
1. Objectives: The learning outcome described on page 3 of this report is aligned closely with actual learning outcomes specified in English 103 and 104 course outlines. The revised wording of this outcome is congruent with recommendations made by the Senate Executive Committee on November 30, 2005.
2. Controlling of longitudinal data for SAT and ACT scores: The committee has determined that SAT and ACT scores are sufficient for controlling longitudinal data. Therefore, neither EPT scores nor EAP (Early Assessment Program) are necessary for this purpose.
3. Use of grades as longitudinal data: The committee affirms its commitment to the use of grades in “approximately six to eight writing intensive GE courses taken after the completion of English 103 and 104.” Performance in writing intensive courses, when controlled for SAT and ACT data, is an appropriate indicator of students’ writing skills after the completion of English 103 and 104.
4. Scoring Rubric: In response to the question about the choice of a scoring rubric, the committee has selected the EPT scoring rubric (see following Addendum), a scale appropriate for the EPT prompts to be used in the spring reading. An additional reason for this choice is that the EPT scoring rubric has been validated by the Educational Testing Service.
Responses to other comments (specificity of methods, identity of essay readers, choice of spring reading): The committee stands by the basic structure of the assessment plan, and it looks forward to working out additional details (specific GE writing intensive courses to be selected, identity of specific faculty to participate in the spring reading) once approval of the plan is secured. As to the timing of the essay reading, the choice of the spring quarter was made in response to particular circumstances associated with the 2005-2006 academic year. If similar assessments are to be carried out in the future, the timing of such readings will be carefully considered with respect to issues of representativeness and validity.
Addendum: EPT Scoring Guide

CSU English Placement Test Scoring Guide
At each of the six score points for on-topic papers, descriptors of writing performance are

lettered so that:


a. = response to the topic

b. = understanding and use of the passage

c. = quality and clarity of thought

d. = organization, development, and support

e. = syntax and command of language

f. = grammar, usage, and mechanics


Score of 6: Superior
A 6 essay is superior writing, but may have minor flaws.
A typical essay in this category:
a. addresses the topic clearly and responds effectively to all aspects of the task

b. demonstrates a thorough critical understanding of the passage in developing an insightful

response

c. explores the issues thoughtfully and in depth

d. is coherently organized and developed, with ideas supported by apt reasons and well-chosen

examples


e. has an effective, fluent style marked by syntactic variety and a clear command of language

f. is generally free from errors in grammar, usage, and mechanics



Score of 5: Strong
A 5 essay demonstrates clear competence in writing. It may have some errors, but they are

not serious enough to distract or confuse the reader.


A typical essay in this category:
a. addresses the topic clearly, but may respond to some aspects of the task more effectively

than others

b. demonstrates a sound critical understanding of the passage in developing a well-reasoned

response


c. shows some depth and complexity of thought

d. is well organized and developed, with ideas supported by appropriate reasons and examples

e. displays some syntactic variety and facility in the use of language

f. may have a few errors in grammar, usage, and mechanics


Score of 4: Adequate
A 4 essay demonstrates adequate writing. It may have some errors that distract the reader, but

they do not significantly obscure meaning.


A typical essay in this category:
a. addresses the topic, but may slight some aspects of the task

b. demonstrates a generally accurate understanding of the passage in developing a sensible response

c. may treat the topic simplistically or repetitively

d. is adequately organized and developed, generally supporting ideas with reasons and examples

e. demonstrates adequate use of syntax and language

f. may have some errors, but generally demonstrates control of grammar, usage, and mechanics


Score of 3: Marginal
A 3 essay demonstrates developing competence, but is flawed in some significant way(s).
A typical essay in this category reveals one or more of the following weaknesses:
a. distorts or neglects aspects of the task

b. demonstrates some understanding of the passage, but may misconstrue parts of it or make

limited use of it in developing a weak response

c. lacks focus, or demonstrates confused or simplistic thinking

d. is poorly organized and developed, presenting generalizations without adequate and

appropriate support or presenting details without generalizations

e. has limited control of syntax and vocabulary

f. has an accumulation of errors in grammar, usage, and mechanics that sometimes interfere

with meaning
Score of 2: Very Weak
A 2 essay is seriously flawed.
A typical essay in this category reveals one or more of the following weaknesses:
a. indicates confusion about the topic or neglects important aspects of the task

b. demonstrates very poor understanding of the main points of the passage, does not use the

passage appropriately in developing a response, or may not use the passage at all

c. lacks focus and coherence, and often fails to communicate its ideas

d. has very weak organization and development, providing simplistic generalizations without

support


e. has inadequate control of syntax and vocabulary

f. is marred by numerous errors in grammar, usage, and mechanics that frequently interfere

with meaning
Score of 1: Incompetent
A 1 essay demonstrates fundamental deficiencies in writing skills.
A typical essay in this category reveals one or more of the following weaknesses:
a. suggests an inability to comprehend the question or to respond meaningfully to the topic

b. demonstrates little or no ability to understand the passage or to use it in developing a

response

c. is unfocused, illogical, or incoherent

d. is disorganized and undeveloped, providing little or no relevant support

e. lacks basic control of syntax and vocabulary

f. has serious and persistent errors in grammar, usage, and mechanics that severely interfere

with meaning


Readers should not penalize ESL writers excessively for slight shifts in idiom, problems with

articles, confusion over prepositions, and occasional misuse of verb tense and verb forms, so



long as such features do not obscure meaning.


 The prerequisite for English 103 is English 102; both courses together are equivalent to English 104. Students for whom English is a second language will enroll in English 102 if their EPT scores do not meet the minimum standard required for placement in 104.




The database is protected by copyright ©sckool.org 2016
send message

    Main page