Tuesday, Oct 25, 2005 (9:15 AM - 10:15 AM) Coral Tower, Lobby Level
AES, AI, ACCUPLACER/WritePlacer
When essays are scored by human experts, the scoring characteristics can be mapped by Artificial Intelligence (AI) and used in Automatic Essay Scoring (AES).
AI is used to identify and internalize essay features into scoring models (algorithms).
The algorithms are verified in simulation and subsequently on live essays.
The algorithms are used by AES to score an essay.
Automatic Essay Scoring
AI maps salient characteristics of freshman essays (about 300) into a linear model of each score (for example, 6s ; 8s ; 10s, etc.)
AES is carried out by mathematically matching live essays to these predetermined linear models to predict a score.
AES algorithms specify whether an essay’s characteristics mathematically match the semantic space previously specified by human graders.
AES therefore emulates human raters by repeatedly evaluating characteristic essay features such as Structure, Content, Style, Syntax, Discourse, and Word choice to predict a maximum likelihood estimate of a score according to the algorithms copied from the 300 human-expert scored essays.
AES’s performance has been verified in national level studies and now waits for users to conduct performance tests at local levels.
We conducted our local performance study with ACCUPLACER/WritePlacer.
WritePlacer employs AI called Intellimetrics
WritePlacer infers and internalizes the rubric and pooled judgments of human scorers by analyzing over 300 semantic, syntactic and discourse features in five categories: