Strategies to evaluate competence christina B. DeBiase, Edd wvu school of Dentistry steps in evaluation



Download 9.77 Kb.
Date12.05.2017
Size9.77 Kb.
#19423

STRATEGIES TO EVALUATE COMPETENCE Christina B. DeBiase, EdD WVU School of Dentistry

STEPS IN EVALUATION

  • Define the objectives of the learning activity – a test should evaluate the degree of student learning
  • Develop a test blueprint

TEST BLUEPRINTS

  • How much emphasis should be given to each objective?
  • What type of question would best assess the students’ attainment of objectives?
  • How difficult should the questions be to adequately discriminate the stronger students from the weaker students?
  • What considerations should be given to test format and administration?

Basic Forms of Evaluation

  • Objective – data gathered represents some predetermined standard
  • Subjective – no prior standard of performance has been specified

METHODS FOR EVALUATING STUDENTS

  • TESTS
  • CBL / PBL SCENARIOS
  • WRITTEN & ORAL PRESENTATIONS
  • SIMULATIONS
  • PRACTICUMS
  • PORTFOLIOS
  • OSCE
  • ETC.

OPTIONS FOR TEST ITEMS

  • Stand-alone multiple-choice items
  • Case-based multiple-choice items
  • True/False
  • Fill-in the blank/ Short-answer/Completion
  • Essay
  • Listing
  • Oral
  • Rating sheets

COMMON TEST-WRITING ERRORS

MULTIPLE-CHOICE COMPOSITION

  • STEM: SET UP THE PROBLEM OR SET OF CIRCUMSTANCES; SHOULD HAVE AT LEAST ONE VERB; CAN BE WRITTEN IN QUESTION FORM OR AS A PARTIAL STATEMENT
  • DISTRACTORS: ONE CORRECT OR BEST RESPONSE (tests higher level learning) AND 3-4 ALTERNATIVES; ALL OF THE ABOVE OR NONE... ARE NOT SPECIFIC OPTIONS OR SOLUTIONS AND SHOULD BE AVOIDED

KEEP IT SIMPLE!

  • DO NOT PUT EXTRA, UNNECESSARY INFORMATION IN THE STEM: Mary, a 16-year old female with cerebral palsy, presents with a bruxing habit. Bruxism is the rhythmic grinding of teeth that can cause overdevelopment of a muscle of mastication. Which of the following muscles may be overdeveloped in a patient with bruxism?

POSITIVE/NEGATIVE STEMS

  • POSITIVE - Usually more effective because the item tests what examinees know to be correct. Which of the following antibiotics is recommended for SBE coverage?
  • NEGATIVE - Sometimes appropriate to distinguish the incorrect response among several correct alternatives. All of the following congenital cardiac defects require antibiotic premedication prior to dental procedures except one. The EXCEPTION is ___?

RULES TO FOLLOW WHEN CREATING ALTERNATIVES

  • INCLUDE REPITITIOUS WORDS IN THE STEM TO AVOID REPEATING WORDS IN EVERY ALTERNATIVE.
  • ALTERNATIVES SHOULD BE SIMILAR IN CONSTRUCTION AND OF APPROXIMATELY EQUAL LENGTH
  • DISTRACTORS SHOULD BE PLAUSIBLE & MAY REPRESENT COMMONLY HELD MISCONCEPTIONS ABOUT A SUBJECT
  • AVOID RESPONSE PATTERNS

The correct answer is longer or more specific than the distractors:

  • All of the following determine the latent period’s degree of biologic effect except one. The EXCEPTION is ___?
  • a. The exposure time
  • b. The type of radiation
  • c. The type of tissue irradiated
  • d. The ability to cause substances to fluoresce *
  • HINT- Put “The” in the stem!

ADDITIONAL PRACTICES TO TO AVOID WHEN DESIGNING ALTERNATIVES FOR MULTIPLE-CHOICE QUESTIONS

The alternatives are not mutually exclusive (overlapping alternatives); if numbers, always list in ascending or descending order & if ranges, intervals should be equivalent: Which of the following I.Q. ranges is descriptive of a person with moderate mental retardation? a. 50-70 b. 35-50 c. 20-35 d. <20

Distractors are not plausible: Which of the following primary molars is a smaller replica of the permanent first molar: a. First b. Second c. Third

Grammatical clues in the stem suggest the correct answer:

  • Which of the following drugs is an antihypertensive agent?
  • a. Procardia
  • b. Prednisone
  • c. Cyclosporine and Imuran
  • d. Amoxicillin and Erythromycin

Information in the stem is also found in the correct answer:

  • Which of the following forms of diabetes requires insulin for effective management?
  • a. Insulin - dependent
  • b. Non-insulin dependent
  • c. Hyperglycemia
  • d. Hypoglycemia

CAUSE AND EFFECT ITEMS

  • (Statement)Prior to cardiac surgery, a patient should receive a dental cleaning, (Reason) BECAUSE in a plaque-infected oral cavity the likelihood of bacteremia is greatly increased.
  • a. Both the statement and reason are correct and related.
  • b. Both the statement and the reason are correct, but NOT related.
  • c.The statement is correct, but the reason is NOT.
  • d.The statement is NOT correct, but the reason is correct.
  • e. NEITHER the statement NOR the reason is correct.

PAIRED TRUE & FALSE ITEMS

  • I. Medical record entries should be completed in non-erasable ink. II. If a mistake is made when making a notation, the incorrect information should be covered with correction fluid.
  • A. Both statements are true.
  • B. Both statements are false.
  • C. The first statement is true, the second is false.
  • D. The first statement is false, the second is true.

ADVANTAGES OF MULTIPLE-CHOICE ITEMS

  • Flexible
  • Measure student’s knowledge in a subject area without the influence of writing skills, spelling, or grammar
  • Scored quickly
  • Content of course can be adequately sampled
  • Comparable results can provide diagnostic information about class
  • The best answer does not have to be the only correct response, but should be defensible
  • Item- analysis is very feasible

DISADVANTAGES

  • Questions are difficult to develop with sufficient plausible alternatives
  • Cannot measure higher level learning unless case-based; stand-alone items usually measure factual recall only
  • Guessing the correct answer can make the test ineffective

GUIDELINES FOR TRUE & FALSE STATEMENTS

  • AVOID:
  • Negatively worded statements
  • Testing trivial detail
  • Specific determiners that act as clues – usually, never, always, all

GUIDELINES FOR MATCHING ITEMS

  • Specify whether alternatives can be used once or more than once
  • It is recommended to provide more alternatives than premises, but the number of alternatives should not greatly exceed the premises
  • List premises or phrases on the left and alternatives on the right

GUIDELINES FOR COMPLETION QUESTIONS

  • Include sufficient information to tell student what type of answer is expected
  • Omit as few words as possible
  • Use a uniform blank space
  • Avoid grammatical clues
  • Place the blank toward the end of the sentence

GUIDELINES FOR ESSAY QUESTIONS

  • Cover the objectives
  • Prepare a structured response with elements students should include for credit
  • Give clear directions so that the student knows what is expected
  • Ensure the test is short enough to allow time to complete the essay(s)
  • Avoid testing trivial details
  • Avoid giving students the option to answer 5 of 7 essays , for example
  • Grade by reading all students’ responses to one essay before going on to the next question


Download 9.77 Kb.

Share with your friends:




The database is protected by copyright ©sckool.org 2022
send message

    Main page