How occurrences of nonstandard English influence the evaluation of written work in schools is surprisingly understudied.
This study seeks to discover the relative weights assigned to two linguistic features associated with good writing and to occurrences of nonstandard English, in this case, morphosyntactic features of African American English (AAE).
We selected an essay from a website, (http://www.emsc.nysed.gov/ciai/dbq/assmtexp.html),
prepared by the Social Studies Division of the New York State Education Department. It was given the highest possible ranking for a 12th grade essay. The topic was democratic versus totalitarian forms of government.
We then modified this “good essay” (GE) lexically and syntactically, resulting in a “bad essay” (BE) (as in Freedman 1978).
We substituted less sophisticated vocabulary or paraphrases. Typical substitutions were as follows:
Using average words per “T-unit” (roughly, independent clauses with associated subordinate clauses) as a measure of sophistication, based on Hunt (1965), we did away with many complex grammatical constructions, reducing words per T-unit from 14.6 to 9.3. (Hunt’s work showed that average words per T-unit for 12th graders is 14.4, 8th graders 11.5, and 4th graders 8.6).
Typical syntactic modifications
For example, in the change from
“In a totalitarian form of government, the citizens have no say in the decisions that are made.”
“There are forms of government with only one person in charge. The citizens have no say in the decisions that are made.”
there is a replacement of the lexical item “totalitarian” with a paraphrase and a reduction of a complex sentence with a word per T-unit score of 17 to two with scores of 11.
The following 8 AAE morphosyntactic features were added to both the GE and the BE
These same AAE constructions were then doubled in both texts (which we called “heavy” AAE use), resulting in six different essays:
GE-NoAAE (Good essay; no AAE)
GE-LAAE (Good essay; light AAE)
GE-HAAE (Good essay; heavy AAE)
BE-NoAAE (Bad essay; no AAE)
BE-LAAE (Bad essay; light AAE)
BE-HAAE (Bad essay; heavy AAE)
These six essays were presented to undergraduate students at Michigan State University at the beginning of Fall Semester 2005 (N=173).
The majority (N=93) are English majors and have secondary English teaching as their goal. The remaining respondents are mostly English and History majors. Demographic information (age, sex, ethnicity, region) and career goals and relevant experience (e.g., tutoring) was sought for each respondent.
Each student who participated saw only one form of the essay and was asked to grade this essay as if it were the work of a high school senior, using the traditional A-B-C-D-F scale.
The participants were also encouraged to write comments on the essay itself or on the back of the op-scan sheet used to collect the data.
The data were analyzed statistically after converting the A-F scale into a 0.0 - 4.0 scale, one more often associated with university-level work. The 0.0-4.0 scale is used in the reporting of the data that follows, but 4.0 may be equated with an “A” and so on down the scale.
Before we look at the results, let’s imagine some of the possible outcomes:
POSSIBLE OUTCOME #1
Nothing counts for nothing. Everybody gets a “C,” or you can imagine an “A” or an “F” if you like.
POSSIBLE OUTCOME #2
A good essay is rated high and a bad one low, regardless of AAE amount or presence.
Rural respondents are much harsher in their ratings overall (.05), but small numbers in each cell do not allow us to relate that to the quality of the essay or the presence or amount of AAE.
These students, many English teachers in training, are in fact very harsh overall. The “good essay” (with no AAE) got a mark just barely above a “C.” Experienced professionals obviously thought much better of it.
Some Qualitative Results
We hope to shed some light on these quantitative results by surveying some of the qualitative data collected.
For the moment we limit ourselves to some examples of these comments. We plan a more thorough investigation based on content classification.
Here are 3 comments on the worst-rated BadEssay/Heavy AAE
This first respondent refused to assign a grade:
“I stopped at this point [i.e., end of paragraph 2]. There is no thesis, organization, transition or grammar. Please re-write, starting w/ a clear thesis & outline. The thesis needs some serious work! It needs to be more detailed, clearer, arguable etc…”
[The respondent liberally annotated the essay with comments and corrections up to that point.]
“Your paper did seem to get better as I read on but it needs a lot of work. I would love to meet with you after school & discuss the corrections I have made. –Mrs. Teacher”
[She corrected all AAE and offered several other suggestions which would have made the grammatical structure more complex.]
Grade: Grammar: D, Overall: C-
“Overall, first para. is to [sic] wordy, needs better thesis. Overusing the word this & they. PS, Is this a real essay?”
[This rater also wanted the essay double spaced and the first paragraph indented.]
And here is a comment made on a Good Essay/LightAAE
“If there weren’t so many grammatical errors the essay would have recieved [sic] an “A”, but the errors were sufficient enough to lower the grade by a whole letter.”
Finally, here are 3 comments on the GoodEssay/NoAAE
The respondent made the notation “first name” by “Mussolini.”
“avoid ‘good’ and ‘bad’ use high school vocabulary”
“What a steaming pile of crap! This is one of the most boring papers I have ever read.” [Other expurgated comments.]
Some comments found in several essays included
Misidentification of many short sentences as “sentence fragments” (e.g., “This causes them to be unhappy.”).
Complaints that the essay contained “incorrect information,” although the essay was written by a student who had been given a reading on these facts.
In conclusion, English teachers-to-be (at least at Michigan State University) are very fussy graders, but attention to AAAE grammatical features does not overwhelm their evaluation of these essays in general, although an interaction between heavy AAE use and a less sophisticated essay (with regard to vocabulary and structure) results in a statistically lower evaluation.
What would real teachers (not teachers in training) do? Would those in other areas (e.g., history, social studies) have rated these essays differently. What is the source of the rural-urban difference?
Would it have made any difference if we had told the respondents that this was “on-the-spot” work? We noted only that it was 12th grade written work; some comments suggested that extensive proof-reading and revision would have helped.
3) What other characteristics of good and bad essays or other nonstandard features might be studied in this format? How important would they prove?
4) Would further demographic differences of raters (whether trainees or teachers) make any difference? Our respondent group was almost uniquely European American.
We hope to have more to report to you along the lines of these improvements and others which we hope to get from you. Thank you for your attention. You can see this PowerPoint and the complete text of all six essays at www.msu.edu/~preston