IntelliMetric® Assessment Tools and Online Essay Grading
- IntelliMetric® Assessment Tools and Online Essay Grading
- IntelliMetric® is an intelligent automated essay scoring system that:
- Provides immediate holistic and analytic feedback for extended or short answers
- May be used as an instructional tool or for high stakes assessment
- Is based on more than a decade of research and development with continual innovations and advancements
- Was first commercially used in 1998 for use in college course placement assessments
- Who’s Using IntelliMetric®?
- Association of American Medical Colleges (AAMC) for the writing section of the Medical College Admission Test (MCAT).
- Graduate Management Admission Council (Analytical Writing Assessment (AWA) portion of the GMAT).
- The College Board (ACCUPLACER® WritePlacer and SAT Prep).
- ACT, Inc. (COMPASS e-Write™, COMPASS ESL, ACT Prep, WorkKeys ®).
- Medical College Admission Test (MCAT)
- State Departments of Education and School Districts throughout the country.
- Creates a unique scoring solution (model) for each prompt
- Learns the characteristics of the score scale through exposure to examples of previously-scored responses
- Uses Artificial Intelligence to learn how to score essays
- Trained with a set of known papers (previously scored by experts)
- Modeled after traditional expert scoring process
- How does IntelliMetric® work?
- Prompt specific training mirroring the human scorer calibration process:
- How is IntelliMetric® trained?
- IntelliMetric® infers rubric and learns the characteristics of the score scale from training set
- IntelliMetric® is inductive rather than rule based
- Model is created and used to score validation set
- Performance data is reviewed (comparing human score and IM score for validation set)
- Agreement Rates
- Pearson correlation (measure of linear relationship)
- Continued……How is IntelliMetric® trained?
- How is an IntelliMetric® model created?
- Hundreds of studies provide support that IntelliMetric®:
- Agrees with expert scoring more often than two skilled human raters will agree with each other.
- Accurately scores open-ended responses across a variety of grade levels, subject areas and contexts
- IM is perfectly internally consistent. It will consistently score the same essay the same way every time. This is not true of human scorers.
- Shows stable results across samples.
- What does the data say about the quality of the scoring?
- Uses over 400 semantic, syntactic, and discourse-level features
- Neurosynthetic approach
- Learning engine
- Uses Multiple “Judges”
- Other automated essay scoring systems use a more narrow approach.
- What makes IntelliMetric® unique?
- Let’s take a look at how IntelliMetric® works.
The database is protected by copyright ©sckool.org 2016