In addition to the studies described above with university students and well-known software applications, we will elaborate on a study in upper secondary schools with a new environment. In the COSAR project (Erkens, Prangsma, Jaspers, & Kanselaar, 2002) we developed the groupware program TC3 (Text Composer, Computer supported & Collaborative) with which the students carry out the main writing task. This environment is based on an earlier tool called CTP – Collaborative Text Production (Andriessen, Erkens, Overeem, & Jaspers, 1996), and it combines a shared text editor, a chat facility, and private access to a notepad and to information sources to encourage collaborative distance writing. The participants worked in pairs within TC3, each partner working at his/her own computer, and wherever possible partners were seated separately in different classrooms.
The main screen of the program displays several private and shared windows. The basic environment, shown in Figure 3.3, contains four main windows:
The upper half of the screen is private and the lower half is shared.
INFORMATION (upper right window): This private window contains tabs for the assignment, sources and TC3 operating instructions. Sources are divided evenly between the students. Each partner has 3 or 5 different sources plus one – fairly factual – common source. The content of the sources cannot be copied or pasted.
NOTES (upper left window): A private notepad where the student can make non-shared notes.
CHAT (lower left, 3 small windows): The student adds his/her chat message in the bottom box: every letter typed is immediately sent to the partner via the network, so that both boxes are WYSIWIS: What You See Is What I See. The middle box shows the incoming messages from the partner. The scrollable upper chat box contains the discussion history.
SHARED TEXT (lower right window): A simple text editor (also WYSIWIS) in which the shared text is written while taking turns.
Figure 3.3: The layout of the interface of the TC3-basic environment.
Text from the private notes, chat, chat history and shared text can be exchanged through standard copy and paste functions. To allow the participants to adjust their focus between their private work and the collaboration, three layout buttons were added in the left-hand corner: the middle layout button enlarges the private windows, the rightmost button enlarges the shared windows, and the leftmost layout button restores the basic layout. The buttons search; mark and delete (zoek, markeer and wis) can be used to mark and unmark text in the source windows and to search through the marked texts. The number of words (aantal woorden) button allows the participants to count the number of words in the shared text editor at any given moment. The stop (stoppen) button will end the session. The traffic light button serves as the turn taking device necessary to take turns in writing in the shared text editor.
In addition, two planning modules were developed in the TC3 program for the experimental conditions: the Diagram and the Outline. The Diagram (see Figure 3.4) is a tool for generating, organizing and relating information units in a graphical knowledge structure comparable to Belvédère (Suthers, Weiner, Connelly, & Paolucci, 1995; Suthers, & Hundhausen, 2001). The tool was conceptualized to the students as a graphical summary of the information in the argumentative essay. Students were told that the information contained in the Diagram had to faithfully represent the information in the final version of their essay. We hoped that this requirement would help students to notice inconsistencies, gaps, and other imperfections in their texts, and encourage them to review and revise. In the Diagram, several types of text boxes can be used: information (Informatie), position (Standpunt), argument pro (Voorargument), support (Onderbouwing), argument contra (Tegenargument), refutation (Weerlegging), and conclusion (Conclusie). Two types of connectors were available to link the text boxes: arrows and lines. The Diagram can be used to visualize the argumentative structure of the position taken.
Figure 3.4: The Diagram Window in the TC3 program.
The Outline (see Figure 3.5) is a tool in the TC3 program for generating and organizing information units as an outline of consecutive subjects in the text. The Outline tool was designed to support planning and organization of the linear structure of the texts. The tool was designed to allow students to construct an overview or hierarchical structure of the text to be written, to help in determining the order of content in the text. In addition, the Outline tool has the didactic function of making the user aware of characteristics of good textual structure, thus allowing the user to learn to write better texts. The Outline has a maximum of four automatically outline numbered levels. Both planning windows are WYSIWIS.
Figure 3.5: The Outline window in the TC3 program.
The effects of the organizer (Diagram) were expected to be related mostly to the consistency and completeness of the knowledge structure in the text (Veerman & Andriessen, 1997). The effects of the lineariser (Outline) were expected to be related mostly to the persuasiveness of the argumentation and the adequate use of language in the shape of connectives and anaphora (Chanquoy, 1996). We expected these effects to take place especially when both organization and linearization were supported, and explicit attention was being paid to translating the conceptual structure into the linear text. The main indicators of this would be increasing attention to the opposite position, and the use of counterarguments. A help facility, the Advisor, gave advice on how to use the Diagram and Outline tools.
In order to compare the effects of the planning tools on the process of collaborative argumentative writing a (quasi) experiment was set up varying the different combinations of planning tools. The effect of the tools on collaborative writing were investigated in the experimental conditions shown in Table 3.1.
The participants were 290 Dutch students aged 16 to 18 from six secondary schools. 145 randomly assigned pairs were asked to write an argumentative text of about 600 to 1000 words defending a position on cloning or organ donation. The shared text had to be based on information sources given within the groupware program.
All communication and activities during the collaboration were logged automatically in a chat and activity protocol. It is possible to replay a whole session with all the keyboard input, including typing errors, deletions, mouse clicks, etc. on the basis of the log file.
The main task in this study was a collaborative writing task. The assignment was to write an argumentative text on cloning or organ donation. For organ donation each partner had five private sources plus one common source, so there were eleven sources in total. The sources were taken from the Internet sites of Dutch newspapers. The assignment was to convince the Minister of Health, Welfare and Sport of the position they had taken. For cloning the partners each had three sources and one common source, so there were seven sources in total. In all groups, partners were seated in separate computer rooms, to encourage them to communicate only through TC3.
The assignment was completed in two to six sessions with an average total duration of 3.9 hours.
3.3.3 MEPA: a Tool for Multiple Episode Protocol Analysis
We use the program MEPA to analyze all the data the students produce in the TC3 environment. The purpose of MEPA2 (Multiple Episode Protocol Analysis), a program for protocol analysis, is to offer a flexible environment for creating protocols from verbal and non-verbal observational data, and annotating, coding and analyzing these.
The program is multifunctional in the sense that it allows for development of both the coding and protocolling systems within the same program, as well as direct analysis and exploration of the coded verbal and non-verbal data using several built-in quantitative and qualitative methods of analysis. In its current version, MEPA can execute frequency and time-interval analyses; construct cross-tables with associative measures; perform lag-sequential analysis, interrater reliability, visual, word frequency and word context analyses; and carry out selecting, sorting and search processes. Also, some aids for inductive pattern recognition have been implemented. MEPA uses a multidimensional data structure, allowing protocol data to be coded on multiple dimensions or variables. To minimize the work associated with coding protocols and to maximize coding reliability, MEPA contains a module that can be used to program complex structured if-then rules for automatic coding. Figure 3.6 shows a screen dump of the MEPA program.
Figure 3.6: MEPA program for protocol analyses.
3.3.4 Analysis of the Argumentative Texts
Each of the 145 student pairs produced one text, and these were analyzed on several dimensions. As a preparation for the final assessment, the texts were imported into MEPA, with a single sentence – defined by a period – per line. The sentences with potentially multiple argumentative functions were split into smaller units using an automatic splitting filter, so that the constituents of sentences such as “Cloning is good, but it can also have side effects” could be properly coded as position and argument contra. The sentences were split automatically where necessary on the basis of argumentative and organizational markers, such as but, however, although, therefore, unless. Before coding, the experimenters manually divided the final texts into segments, largely based on the existing paragraph structure. The final argumentative texts were scored on five variables.
Table 3.2: Description of text quality measures.
The formal structure of the text as defined by introduction, body, and conclusion.
The quality of the argumentation within the paragraphs.
The quality of the main line of argumentation in the text.
The presentation towards the reader and the level of formality of the text.
Mean text score
The mean of the four scores above.
3.3.5 Analyses of the Chats
The chat protocols were not analyzed at a propositional level like the argumentative texts, but rather at an episode level based on the task oriented collaboration process.
The chat protocols were manually divided into episodes of different Task act categories. Whenever the focus of the discussion changed within a particular type of Task act, a new episode was started as well. In addition, MEPA automatically coded a new episode whenever the partners had not used the chat window for more than 59 seconds.