How do people learn to perform new actions through observation? Does it involve the immediate formation of a new motor representation of the action, or is more abstract thought, such as mentally rehearsing spoken instructions, involved in learning the new action? This study aims to answer these questions by examining how the rolandic Mu rhythm (8-14 Hz), found in the human electro-encephalogram, is influenced by action sequence learning by observation. The Mu rhythm, found over the human motor cortex, is known to be suppressed by arm and hand movements, as well as by the observation of others performing similar movements. Electro-encephalograms were measured to determine Mu rhythm power in two conditions. During an imitation condition participants observed sequences of button presses presented on a computer, with each sequence presented four times. Participants were required to reproduce each sequence immediately after the fourth presentation. During the detection condition participants observed action sequences similar to the ones used during the imitation condition, but only had to detect rare deviant button presses, where buttons on screen were pressed with a thumb instead of an index finger. The detection condition served as a control condition where the Mu rhythm might already have been somewhat inhibited by action observation, which allowed the full effect of learning by observation from the imitation condition to be examined. Relative to the observation of sequences in the detection condition, Mu power was reduced during the observation of sequences in the imitation condition, indicating an increased processing load in the motor cortex. This suggests that action sequence learning by observation involves the formation of new motor representations of such action sequences. Furthermore, Mu power was reduced the most during observation of the second, third and fourth presentations of each sequence in the imitation condition, with a stronger Mu rhythm visible during the initial presentation. This indicates an additional processing load in the motor cortex after the initial presentation, which could mean that motor representations are rehearsed after the initial formation, suggesting that a certain amount of abstract thought might also be present while learning to perform new actions.
The learning of action sequences by observation is a well-studied subject in cognitive psychology. It has been used in a wide variety of fields, including the study of amnesia (Adlam, Vargha-Khadem, Mishkin, & De Haan, 2005), schizophrenia (Delevoye-Turrell, Giersch, Wing, & Danion, 2007), the development of motor skills such as tool use (Järveläinen, Schürmann, & Hari, 2004) and theories about the evolution of language (Molnar-Szakacs, Kaplan, Greenfield, & Iacoboni, 2006; Muthukumaraswamy, Johnson, & McNair, 2004). However, opinions are still divided on the exact way in which action sequence learning by observation works.
Baddeley’s (2003) model of working memory states that the working memory consists of four components: a visuospatial sketchpad, used for storing visual imagery; a phonological loop, which is a short-term storage based on sound and language; an episodic buffer which binds together information to created integrated episodes; and a central executive which divides attention between all these components, and can not only recall old memories, but can turn the information contained within into new representations. Applied to learning a new action sequence by observation, this model would suggest a rather abstract way of learning. An observed action sequence could be stored in the visuospatial sketchpad, perhaps converted into language using the phonological loop, before being recombined into an episodic representation which can be rehearsed in order to learn how to perform the new action sequence. As an example, a sequence of button presses could first be observed, with each button press being converted into mentally spoken text, before these are mentally combined (“first press the middle button, then the right, then the left”) in order to learn how to perform the complete action sequence. This type of abstract thought would generally be accompanied by increased activity in the frontal cortex of the brain (e.g. Miller & Cohen, 2001).
Another possibility is that observed action sequences are more directly mapped onto motor representations of those sequences, using available pathways in the motor areas of the brain. In their magnetoencephalography (MEG) experiment, Van Schie, Koelewijn, Jensen, Oostenveld, Maris and Bekkering (2007) found near immediate activation of the motor cortex after observation of a hand pushing a button from an egocentric perspective. Their findings suggest the existence of a process linking observed hand movements directly to the motor cortex. The this process could possibly be further facilitated through the use of mirror neurons, which are neurons which fire both when people observe an action being carried out by another and when they carry out the action themselves (Cochin, Barthelemy, Roux, Martineau, 1999; Iacoboni, Woods, Brass, Bekkering, Mazziotta, & Rizzolatti, 1999; Falck-Ytter, Gredebäck, & Von Hofsten, 2006). It is possible that learning how to reproduce an action sequence after observing it works in a similar way, evoking increased activity in areas such as the motor cortex as well. Activity in this area can be indexed by measuring the amplitude of the Mu rhythm, present in the human scalp electro-encephalogram (EEG). The present study will attempt to reveal the mechanisms behind action sequence learning by observation through analyzing this Mu rhythm.
The rolandic Mu rhythm, usually encompassed in the Alpha and low Beta ranges (8-14 Hz), originates from the human sensorimotor cortex. Originally thought to occur infrequently, more recent studies have shown that it is present in the scalp EEG of most adults (Makeig, Westerfield, Jung, Enghoff, Townsend, Courchesne, & Sejnowski, 2002). It is typically observed when the motor cortex is in a state of rest. If the motor cortex becomes desynchronized, such as when hand or arm movements are made, or even just imagined (Nair, Purcott, Fuchs, Steinberg, & Kelso, 2003), the typical result is diminished power along the Mu rhythm band (Pfurtscheller, Neuper, Andrew, & Edlinger, 1997; Pfurtscheller, Neuper, & Krausz, 2000).
The motor cortex is also known to be engaged by the observation of actions performed by others (Hari, Forss, Avikainen, Kirveskari, Salenius, & Rizzolatti, 1998). This in turn results in a decrease in Mu rhythm power similar to, but to a lesser degree than actual movement execution (Muthukumaraswamy & Johnson, 2004). For example, in their experiment, Muthukumaraswamy and Johnson let participants watch an experimenter who made various hand movements in front of them, which caused a decline in Mu rhythm power. Furthermore, observation of a precision grip caused a statistically significant change in power compared to observation of a simple hand extension, demonstrating that the Mu rhythm is sensitive to subtle changes in observed actions. These results are further strengthened by the finding that goal-directed actions cause a greater desynchronization of neuron populations in the motor cortex than observation of nongoal-directed actions (Järveläinen et al., 2004). This is reflected in a greater Mu rhythm power decrease during goal-directed action observation than during nongoal-directed action observation (Muthukumaraswamy et al., 2004).
These previous findings allow us to form a baseline for examining the influence of sequence learning by action sequence observation on the human Mu rhythm. The present study sets this baseline by recording Mu rhythm power during passive action sequence observation with the help of EEG. These data are compared with Mu recordings obtained from an experimental condition where participants are required to observe the same type of action sequences, only with the intention to memorize them and reproduce them afterwards. If action sequence learning by observation works by mapping action sequences onto motor representations of those sequences, the increased motor cortex activity should cause a significantly inhibited Mu rhythm in the latter condition when compared to the former.
Fifteen participants (six males, nine females) participated in the experiment. Two females were removed during data analysis for methodological reasons. The remaining participants were between 18 and 26 years of age, with a mean age of 20.5 years and a standard deviation of two years. All were right handed as assessed by the Annet Handedness Inventory (Annet, 1970). All participants were university students, gave informed consent to participate and were given course credit as a reward.
A custom response box was constructed for this experiment, consisting of four buttons arranged in a square shape, with a fifth in the middle of this square (see Figure 1a). Each button had a built-in orange LED which could light up. The response box was connected to and positioned in front of a Pentium IV computer with a 17 inch monitor. The experiment itself was programmed and carried out in E-prime (Psychology Software Tools, Inc., http://www.pstnet.com/).
2.3 Stimuli and procedure
Participants sat approximately 70 cm (28 inches) in front of the computer screen. Prior to the start of the experiment, participants were administered a computerized Corsi Block Task (Berch, Krikorian, & Huha, 1998) to assess their visuospatial working memory capacities. This task has often been used as an index for learning at the motor level. As such, a positive correlation of the Corsi Block Task with performance on the current experiment would be a theoretical indication that action sequence learning by observation makes use of motor representations. The Corsi task works by tapping a sequence of blocks in a specific order, which participants were supposed to imitate. In the computerized version these blocks appeared on screen, and participants tapped them by clicking on them with the mouse. Sequence length increased until performance broke down. The final score on the task represents the highest sequence length which was successfully imitated.
After completion of the Corsi Block Task the response box was placed in front of participants at a distance which they found comfortable. The experiment then started, which consisted of an imitation and a detection (control) condition, each containing 40 trials. Each participant carried out the imitation condition first and the detection condition last. The entire experiment, including the application of the electrodes and both conditions, lasted approximately three hours.
In the imitation condition the participants were asked to observe sequences of button presses presented on the monitor, and to reproduce them from memory using the physical response box immediately after they had been presented on the screen for a total of four times. Each trial from the imitation condition consisted of a single sequence repeated on screen four times, followed by the subsequent execution of this sequence by the participant. Sequences were predetermined, but were shown in random order to each participant. Each sequence consisted of six movements and always started with both hands placed in the starting position, meaning that both lower buttons on the response box were depressed using the index fingers. Each movement in a sequence consisted of using one index finger to push one of the upper two buttons, then returning to the starting position. The middle button was never pushed during sequences and was only used as a warning light during sequence presentation, indicating when each individual sequence started and ended. As such, there were four possibilities for each movement in a sequence: moving the left hand to the upper left button and back, moving the left hand to the upper right button and back, moving the right hand to the upper left button and back, and moving the right hand to the upper right button and back.
The exact sequence timing and presentation, as shown on the computer screen, is illustrated in Figure 3. The monitor showed photos of the response box with two virtual hands from an egocentric perspective. Sequence presentation always started with a five second warning signal, so that participants knew that the sequence was about to be shown to them. The warning signal showed the virtual hands in the starting position, with the middle light on the response box shown on the monitor turned on (see the first frame of each sequence in Figure 3). After the on-screen warning signal extinguished, the monitor showed the hands executing all six movements of the current sequence. Each movement was represented by two photos. The first photo showed one of the hands pushing one of the two target buttons, giving the impression that the hand moved to press the button. The second picture showed both hands back in the starting position, resulting in the apparent motion of the hand moving back to the starting position. Each photo was displayed for 500 ms. The entire sequence was repeated three more times (for a total of four sequence presentations) immediately after the previous sequence ended. Each repetition again started with a warning signal, providing a demarcation between the end of one presentation and the start of the next, and a final warning signal was displayed immediately after the last repetition. At this point an instruction screen was displayed, containing text which prompted participants to put both fingers in the starting position, pressing both lower buttons on the response box. This instruction screen also contained a picture showing participants how to place their fingers in the starting position. Pressing the lower buttons caused the warning light in the middle of the physical response box to turn on. The instruction screen told participants to execute the observed sequence from memory as soon as the warning light on the response box extinguished, which it did after keeping both lower buttons depressed for five seconds. At this point participants reproduced the sequence. Before the experiment started they were instructed to execute the sequences as quickly and accurately as possible, but to give priority to accuracy. At the end of each trial, participants received feedback on accuracy, speed and overall progress through the imitation condition. The imitation condition lasted for 40 trials. Participants were also presented with three practice trials at the start of the condition for familiarization purposes, which were not recorded for statistical analysis. The structure itself (six movements per sequence, four sequence presentations and the stimulus timing) was determined in a brief pilot experiment carried out before the present study. During the pilot experiment, participants made one or more mistakes in roughly half of the reproduced sequences, showing that actively learning a sequence is necessary in order to reproduce it correctly.
After completing the imitation condition, participants moved on to the detection condition. During the detection condition participants observed another 40 predetermined sequences, different from the ones used in the imitation condition, but this time they were instructed not to memorize the sequences, but to detect deviant movements where buttons were pressed with a thumb instead of an index finger (see Figure 1b). This condition served as a control condition for the experiment, where participants were actively engaged in action observation, but were not learning any action sequences. Each trial in the detection condition consisted of the presentation of one of the sequences using the exact same timing as was used in the imitation condition previously. Again each sequence consisted of six movements and was presented four times. However, this time there was a one in five chance for each sequence presentation that one of the buttons would be pressed with a thumb instead of an index finger. Participants were not informed about the exact odds of deviant movements occurring, only that they would occur. They were instructed that if they spotted a deviant movement, they were to press any button on the response box during the standard warning signal that followed the sequence presentation. If a mistake was made, the word “FOUT!”, which is Dutch for “WRONG!”, was displayed for the rest of the duration of this warning signal, after which a new, neutral warning signal was displayed, and the experiment continued. At the end of each trial participants received feedback on overall progress and accuracy in the detection condition up to that point. Again, participants were offered three practice trials at the beginning of the condition so that they could become familiarized with the task, but performance on these trials was not recorded for statistical analysis.