Expression of basic emotion on playing the snare drum

Transcription

1 International Symposium on Performance Science ISBN The Author 2011, Published by the AEC All rights reserved Expression of basic emotion on playing the snare drum Masanobu Miura 1, Yuki Mito 2, and Hiroshi Kawakami 3 1 Department of Media Informatics, Faculty of Science and Technology, Ryukoku University, Japan 2 Graduate School of Law, Hitotsubashi University, Japan 3 College of Art, Nihon University, Japan The motions used when expressing emotion in a musical performance are the target of this study, in which a professional percussionist was asked to play while expressing each of five basic emotions, such as anger, happiness, fear, sadness, and tenderness. The characteristic motions for expressing each of the emotions, such as the velocity and height of a drumstick, were then extracted, and these extracted motions were assumed to be the cues used by a player to express emotion. Finally, synthesized motions expressing each emotion were obtained by introducing a geometric average of the obtained motion data (.trc) as the average motion of each emotion. We then conducted an evaluation experiment to reproduce each of the emotions to be recognized and then confirmed that the motions actually represented a particular aspect of the emotional plane. Keywords: motion capture system; snare drum; drumming; emotional expression; performance The motions in musical performances are important as well as its acoustics. In recent years, the myoelectric sensor (Fujisawa and Miura 2010) and the motion capture system have often been used to analyze motion, and studies concerning performance skill have also been conducted (Dahl 2001). However, the motions used when playing with each emotion have not been reported. Therefore, the motions used when expressing an emotion in a musical performance are the target of this study. The motions of a snare drum performance displayed when specified emotion is being expressed were investigated by using a motion capture system. A professional percussionist was

2 630 asked to play the snare drum while expressing five basic emotions and one non-emotion. The relation between the performed and expressed emotions was investigated by observing motion data synthesized from recorded data. The performance cues, such as tempo, vibrato, etc., that are associated with emotion were reported by Juslin and Sloboda (2001). Reports on the motion cues used when playing with emotion have not yet been presented. In Figure 1, the relation between the emotional space reported by Juslin and Sloboda (2001) and performed motion is shown. Outline of measurement MAIN CONTRIBUTION We conducted a recording experiment in which a professional percussionist was asked to play single strokes while expressing each of five basic emotions. The performance task used in the experiment is shown in Figure 2. The specified tempo was 90 bpm, and the length of the performance was one minute. Eight metronome clicks, meaning the length of two bar-lines, were presented to the player prior to the performance. Recording environment The motions of the percussionist were recorded by using a motion capture system at a studio at the College of Art at Nihon University. The motion capture system was a MAC 3D System (Motion Analysis Corp.) recording a frame every 1 /200 s at a shutter speed of 1 /1000 s. The number of markers on the player was 30, as shown in Figure 3. Measuring emotional motion Extracting motions from one bar-length: To obtain the tendencies of the motions displayed during a performance, the motions for the four strokes in one bar-length were recorded. The geometric average for all the recorded motion in one bar-length was then calculated, meaning the average motion, named emotional motion, which was represented as a matrix whose size was 90 (30 points with 3 dimensions) x the number of samples (580 in this example). The emotional motions obtained were Mt (for tender motion), Mh (for happy motion), Ms (for sad motion), Mf (for fear motion), Ma (for anger motion), and Mn (for non-emotional motion). Next, by subtracting geometrically the Mn from each of the emotional motions (or in this case, subtracting the.trc data), we then obtained the differentials for each of the emotional motions, which were Mt', Mh', Ms', Mf', and Ma'.

3 INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 631 <Emotion> Positive valence <Motion>? Low Activity High Activity correspondence?? Negative valence? Figure 1. Relation between emotional space and performed motion. Figure 2. Performance task used in the experiment. Figure 3. Positions of markers on the player. (See full color versions at www. performancescience.org.) Comparing emotional motion: To determine the nature of each motion, we obtained an exaggerated version of each of the motions by multiplying each Mt', Mh', Ms', Mf', and Ma' with five and then it adding the result geometrically to Mn, generating exaggerated motion data for each emotional motion. Although these exaggerated motions were unnatural, they enabled us to see the nature of each emotional motion. Nature of emotional motion: By observing the differences between the exaggerated emotional motions, we found the tendencies of the differences in the height of a stroke, the velocity of an upstroke, and the velocity of a down stroke, as shown in Figure 4(a), (c), and (e). The height and two velocities were obtained by observing constantly the tip of each drum stick for each

4 632 Height of the stroke (pixel) Anger Sad Tender Happy Fear (a) Height of stroke (b) Height of stroke in emotional plane : High : Low Velocity of upstroke (pixel/sample) Anger Sad Tender Happy Fear (c) Velocity of upstroke : Fast : Slow (d) Velocity of upstroke in emotional plane Velocity of downstroke (pixel/sample) Anger Sad Tender Happy Fear (e) Velocity of downstroke : Fast : Slow (f) Velocity of downstroke in emotional plane Figure 4. Nature of each emotional motion for the specified aspects of the drumsticks and the relation of each aspect to the emotional spaces reported by Juslin and Sloboda (2001). (See full color version at frame. In addition, the relation between the emotional space reported by Juslin and Sloboda (2001) and each emotion is shown in Figure 4(b), (d), and (f). Since the tendencies were observed, the aspects for each drumstick are referred to as motion cues, which correspond to the axes in the two-dimensional space that represents emotion.

5 INTERNATIONAL SYMPOSIUM ON PERFORMANCE SCIENCE 633 Height of the stroke : High Velocity of upstroke : Fast Height of the stroke : Low Velocity of downstroke: Slow Height of the stroke : High Velocity of downstroke : Fast Height of the stroke : Low Velocity of upstroke : Slow Figure 5. Relation between the combinations of the motion cues and the emotional planes reported by Juslin and Sloboda (2001). (See full color version at www. performancescience.org.) IMPLICATIONS Using motion cues when expressing emotion through performance By comparing obtained motion cues with two axes, activity and valence, we could see that the axis for activity corresponded to the height of the stroke and the velocity of the down stroke and that the axis for valence corresponded to the height of the stroke and the velocity of the upstroke, shown in Figure 5. Synthesizing motions with specific aspects We synthesized motions with the specific aspects given in Figure 5. We extracted a motion along with one of the denoted motions on the ends of each of the axes in Figure 5 and then obtained the geometric average of the two motions as *.trc data, named synthesized motion. For example, in the case of synthesizing the motion for happiness, we extracted two motions, one with the maximum height of a stroke and the velocity of an upstroke and the other with the maximum height of a stroke and the velocity of a down stroke, so that we could confirm that the motion denoted in Figure 5 actually represented that particular aspect of the emotional plane. Experiment for subjective recognition We conducted an experiment to recognize each five of the basic emotions. We used emotional motion without exaggeration, exaggerated emotional motion, recorded video, and synthesized motion, labelled 1x, 5x, Video, and Synth., respectively. We presented five subjects with 20 patterns of motions (5 emotions x 4 types of stimuli) and asked them to identify an expressed

6 634 Ratio of correct answer [%] x 5x Video Synth. Figure 6. Average and standard deviations of correct ratio of answers by all subjects. emotion. The average of the correct answer, shown in Figure 6, shows that no significant differences were determined by tests such as Tukey s HSD, Dunnett s T3, and Dunnett s C. We found that the synthesized motions correctly represented an emotion with the same accuracy as other stimuli. Juslin and Sloboda (2001) stated that several cues for representing emotion are used differently among players but that they are powerful even though they are communicated in an ambiguous way. This study shows that motion cues exist in a professional percussionist on the basis of the relation confirmed by the synthesized motions. Acknowledgments The author is grateful to Yuki Konishi for her assistance in conducting measurements. This study is partly supported by the Grants-in-Aid for Scientific Research ( ). Address for correspondence Masanobu Miura, Faculty of Science and Technology, Ryukoku University, 1-5 Yokotani, Seta Oe-cho, Otsu, Shiga , Japan; miura@rins.ryukoku.ac.jp References Dahl S. (2001). Arm motion and striking force in drumming. In D. Bonsi et al. (eds.) Proceedings of the International Symposium on Musical Acoustics (pp ). Perugia, Italy: Musical and Architectural Acoustics Laboratory. Fujisawa T. and Miura M. (2010). Investigating a playing strategy on drumming using surface electromyograms. Acoustical Science and Technology, 31, pp Juslin P. N. and Sloboda J. A. (2001). Music and Emotion. Oxford: Oxford University Press.