The implementation of IT-supported examination processes entails organisational changes that are reaching far beyond mere technological aspects. The professional relation between administrative staff (student office and IT administration) on the one hand and scholars and students on the other is profoundly affected by the use of electronic administration processes since Internet-based self service functions allow not only for the administrative staff to process data, but for students as well. Hence, the reorganisation of the examination administration provokes a change process affecting the university as a whole, and, as is generally known, changes frequently evoke opposition by the stakeholders involved. However, there is a gain of efficiency that compensates for these organisational irritations. The general IT support of examinations disburdens the university administration from a multitude of manual operations. Numerous work steps that have been carried out redundantly before such as recording marks become obsolete. Furthermore, access to examination data is provided to all persons involved in the examination process according to a tiered model of roles and authorisations. Further potential for facilitations of work originate from the emergence of computer-based, electronic examinations in higher education. The increasing number of learning management systems at universities has contributed to the popularity and acceptance of electronic assessment systems and on-screen exams. An effective combination and mutual exchange of data between the examination management software, and examination and assessment systems through standardised interfaces will allow for further simplifying the overall examination process in the future. Yet, even without a close linkage between such systems, stand-alone examination tools can reduce costs, and improve the quality and speed of the correction process.
First steps towards computer-based examinations at German universities were taken by e-learning pioneers who have developed cutting-edge examination tools since the late 1990s and have tested a wide range of different exam-related functions. Computer-based examination and test tools have been applied for different purposes, e.g. placement tests, entry-level tests (prognostic tests), selfassessment tests, regular written and oral examinations (selective and diagnostic examinations), and online surveys. Against all odds and substantial barriers, numerous benefits have created a growing interest in on-screen examinations at German universities at this stage
3.1. Benefits and challenges
Today the development of electronic assessment tools and the use of electronic examinations have left behind their initial stage. Admittedly, many critical challenges had to be met on the way: the tremendous efforts that were necessary for the development of electronic exams or for the initial creation of a pool of questions through the lecturers, the insufficient familiarity of users with the new procedures, malfunctioning hardware components or operating systems or a failure in Internet access, legal security requirements that had to be met, insufficient flexibility of examination regulations or attempts at manipulation and fraud. Therefore, the development of a methodologically sound and juridically stable electronic exam scenario demanded overcoming many barriers.
Meanwhile a consolidation with regard to electronic examinations has been accomplished in many although not in all respects. Some of the potentially obstacles related to the transfer of examination procedures from a paper and pencil scenario into a computer-based environment are listed below:
Organisation of on-screen examination settings: planning of simultaneous and of delayed exams, room planning with regard to PC pools and the number of available PC workstations, instruction of staff, establishing test centres, etc.
Conceptual design and methodology of exams: design of examinations and planning of the exam process (generation of test item pools, designing exams along available question types, securing consistent levels of difficulty, etc.), computer-based preparation of the examination (conception and carrying out of practice exams and tests for the purpose of exercising, etc.)
Examination technology and tools: guaranteeing an interference-free and fraud-protected examination process, considering test tool-based intolerance against orthographic or other minor mistakes in the automatic exam correction process, etc.
Judicial requirements: setting up reliable authorisation procedures, guaranteeing a perennial verifiability of results, adding the new exam forms to formal curriculum and exams regulations, etc.
Electronic examinations lead to significant changes in almost all aspects of the organisation of examinations. Benefits and downsides will be reconsidered and weighed out against each other on the basis of the four dimensions of the electronic examination process depicted above.
A main reason for adopting electronic examinations is the possibility to process a significantly larger amount of examinations. Since the Bologna process has multiplied the number of tests that have to be arranged for during the semester, this possibility is of paramount importance for teachers and administrators. Examiners expect a reduction of workload regarding the preparation of exams and the correction process as well as considering administrative tasks such as the digital collection and announcement of examination results. Furthermore, potential sources of error are reduced: problems with the legibility of handwritten exam answers do not apply in the context of digital data processing. Even the administrative processing of exam results is facilitated since media breaks are avoided within a comprehensive electronic environment. Although only a fraction of exam questions
is suited for an automatic correction (i.e. multiple choice, long menu etc.), and although some questions require manual (post-)correction through the examiner, the facilitation of exam corrections generally saves time and increases the correction quality even of complex questions through computer-based pre-correction procedures. From the students’ viewpoint, there are even more benefits of electronic examinations such as new forms of self-contained knowledge diagnostics represented by digital practice exams (for the purpose of exercising) and periodic course-accompanying electronic tests. Self-contained knowledge diagnostics can also be fostered through supplying exemplary solutions to students’ incorrect exam answers. Moreover, the (partial) automatic correction of tests leads to an increase in objectivity of examination marks. Additionally, the notification on results immediately after the end of the exam is highly welcome among students as an effective means of feedback
3.2. Features of electronic examination and testing tools
Many on-screen testing tools (Enlight Candidate, Dynamic Media eTesting, LTMS, NetTest, Questionmark Perception, Respondus, and exam functionalities of learning management systems such as ILIAS, moodle, Stud.IP, Clix, Blackboard etc.) offer a broad range of functions and modules from designing examination questions to data storage:
1) Functions for designing examination and assessment questions such as question and assessment authoring with authoring tools, supporting different question types (yes/no,right/wrong, multiple choice, matching questions, matrix tasks, Likert items, long-menu questions, questions with free-text fields, clozes, including calculated and algorithmic formats and graphics, audio, and video elements), question banking by learning objective, alternatively retrieving exams from e-learning platforms or downloading answer databases for quizzes or surveys, randomising presentation of questions and choices, adaptive branching based on how questions are answered, rapid creation of random sets of questions and assessments through an exam wizard, determining point values and exam settings offline 2) Preview and publish functions such as previewing questions before publishing them to the server, publish examinations, assessments, and surveys directly to one or multiple online courses (batch publishing)
3) Organisational functions and workflow management such as the administration of
software licenses, interfaces for data import, exam participants, etc.
4) Control functions for the exam and the exam setting such as supervising the computer based examination using a role-based security structure for multi-author environments, controlling the exam room, PC monitoring and recording keyboard activities, and
5) Reporting and retrieval features such as giving students instant exam and assessment
feedback to enhance learning, on-demand results delivered to browser, PDA, CD-ROM, or paper, printing and scanning assessments, customizable reports and item analysis, print
exams directly from the exam tool, or save files to standard software, retrieve custom
Naturally, these features do not apply to all available commercial or open-source software systems. Different exam tools offer different profiles and features, have different technical and system requirements and are characterised by different options of integration into existing IT hardware and software infrastructures.
3.3. Lessons learnt at German universities
Since the development and use of electronic examinations still is like entering virgin soil for
numerous German universities, the experiences we have outlined above shall be summed up here and are complemented through several recommendations.
The implementation of electronic examinations comes along with a significant change of the whole academic examination culture. As was shown above, a sustainable success in implementing electronic tests depends upon the overcoming of infrastructural, technical, methodological, and juridical barriers (furnishing adequate PC pools, procurement of adequate software, successful integration into curricula, adaptation of examination regulations, etc.). Due to the increasing amount of examinations related to Bachelor and Master study programmes, the interest in holding examinations and electronic exams more efficiently will stay on the rise. The expected rationalisation effects and the benefits of an immediate feedback make the use of this type of examination attractive for lecturers and students likewise.
Nevertheless, electronic examinations are not appropriate for all bodies of knowledge and scientific disciplines alike due to the limitations of an automatic correction of complex questions and exercises. Therefore, electronic examinations unfold their strengths primarily in assessing basic knowledge in Bachelor programmes (as well as in examinations with huge numbers of students). For regularly recurring examinations a pool of test items with different question formats is advisable. Such a test item pool should be created collaboratively by several lecturers that mutually assess the others’ questions and continuously add new items. A selection of items can be taken from this question pool and can be used for separate exams. A further benefit of on-screen exam tools for students is based upon electronic practice exams and tests for self-contained forms of knowledge diagnostics. Moreover, the use of digital placement tests contributes to a rationalisation of the universities’ admission processes through allowing for a better matching of students’ capabilities on the one hand and study requirements on the other.
Above all, what is needed for a frictionless operation of the new study programmes in the wake of the Bologna process, for coming to terms with a rising number of examinations in some disciplines, for the sustainable reduction of redundancies in data management and for the replacement of dispensable work steps is a tight interlocking of all the different components, institutions and persons that are involved in the examination process chain. This entails a stronger interconnection of IT infrastructures, and particularly a more flexible data exchange between campus management systems, learning management systems, and examination tools.
In addition, the cultural change associated with the implementation of new administration workflows will be put into effect only if substantial reorganisation processes are tackled with foresight. Furthermore, solely academic and administrative staff being positive about the long-term improvements of turning away from highly fragmented traditional examination processes will actually benefit from the disburdening effects supplied by the new electronic examination tools.
Bartram, D. & Hambleton, R. (eds.) (2006). Computer-Based Testing and the Internet: Issues and
Advances. Hoboken, NJ: Wiley.
Käuser, G., Fischer, M.R. & Stieg, M. (2003). Lernen, Lehren und Prüfen mit neuen Medien. In: U.
Beck & W. Sommer (Hrsg.). Learntec 2003. Karlsruhe: KMK GmbH, S. 589-599.
Krischke, W. (2005). Der elektronische Prüfer. In: Die Zeit 32 (04.08.2005).
Parshall, C., Spray, J. A., Kalohn, J. C. & Davey, T. (2001). Practical Considerations in Computer-
Based Testing. Heidelberg: Springer (Statistics for Social and Behavioral Sciences).
Piotrowski, M. & Rösner, D. (2005). Integration von E-Assessment und Content-Management. In: J.
M. Haake, U. Lucke, D. Tavangarian (Hrsg.) (2005). DeLFI2005: 3. Deutsche e-Learning Fachtagung
Informatik der Gesellschaft für Informatik e.V. Bonn: GI-Verlag, S. 129-140.
Roberts, Tim S. (2006). Self, Peer and Group Assessment in E-Learning. Hershey, PA: Information
Ruderich, F. (2004). Computerunterstützte Prüfungen in der ärztlichen Ausbildung nach der neuen