Alle Publikationen
2018
-
(2018): Multicultural Facial Expression Recognition Based on Differences of Western-Caucasian and East-Asian Facial Expressions of Emotions. In: IEICE Transactions on Information and Systems 101 (5), S. 1317-1324. DOI: 10.1587/transinf.2017MVP0025
Abstract: An increasing number of psychological studies have demonstrated that the six basic expressions of emotions are not culturally universal. However, automatic facial expression recognition (FER) systems disregard these findings and assume that facial expressions are universally expressed and recognized across different cultures. Therefore, this paper presents an analysis of Western-Caucasian and East-Asian facial expressions of emotions based on visual representations and cross-cultural FER. The visual analysis builds on the Eigenfaces method, and the cross-cultural FER combines appearance and geometric features by extracting Local Fourier Coefficients (LFC) and Facial Fourier Descriptors (FFD) respectively. Furthermore, two possible solutions for FER under multicultural environments are proposed. These are based on an early race detection, and independent models for culture-specific facial expressions found by the analysis evaluation. HSV color quantization combined with LFC and FFD compose the feature extraction for race detection, whereas culture-independent models of anger, disgust and fear are analyzed for the second solution. All tests were performed using Support Vector Machines (SVM) for classification and evaluated using five standard databases. Experimental results show that both solutions overcome the accuracy of FER systems under multicultural environments. However, the approach which individually considers the culture-specific facial expressions achieved the highest recognition rate.
Keywords: culture specificity, culture specificity of facial expressions of emotions, Emotionserkennung, facial expression recognition, Gestik/Mimik, Mensch-Technik-Relationen (MTR), multicultural FER, Realtechnik, Status / biologische Marker, Technik, universal human tendencies, universality of emotions -
(2018): Die feinen Unterschiede. Kritik der gesellschaftlichen Urteilskraft. Unter Mitarbeit von Bernd Schwibs. 26. Auflage 2018. Frankfurt am Main: Suhrkamp (Suhrkamp-Taschenbuch Wissenschaft)Keywords: Bourdieu, Bourdieus Habitustheorie, Einkommen / Beruf, Familiengeschichte, Favoriten, feine Unterschiede, Feld, Geschmack, Gestik/Mimik, Good manners, Habitus, Klasse, Kleidung, Körperhaltung, Macht, Modelle/Theorien, Rollen, Soziale Angemessenheit, Soziologie, Sprache, Status / biologische Marker
-
(2018): Greetings as social action in Finland Swedish and Sweden Swedish service encounters – a pluricentric perspective. In: Intercultural Pragmatics 15 (1), S. 57-88. DOI: 10.1515/ip-2017-0030
Abstract: While greetings are performed in all cultures and open most conversations, previous studies suggest that there are cross-cultural differences between different languages in greeting behavior. But do speakers of different national varieties of the same language organize and perform their greeting behavior in similar ways? In this study, we investigate the sequential organization of greetings in relation to gaze behavior in the two national varieties of Swedish: Sweden Swedish spoken in Sweden and Finland Swedish spoken in Finland. In recent years, the importance of studying pluricentric languages from a pragmatic perspective has been foregrounded, not least within the framework of variational pragmatics. To date, most studies have focused on structural differences between national varieties of pluricentric languages. With this study, we extend the scope of variational pragmatics through adding an interactional, micro perspective to the broader macro analysis typical of this field. For this study, we have analyzed patterns for greetings in 297 videorecorded service encounters, where staff and customers interact at theatre box offices and event booking venues in Sweden and Finland. The study shows that there are similarities and differences in greeting behavior between varieties. There is a strong preference for exchanging reciprocal verbal greetings, one at a time, in both. There is also a similar organization of the greeting sequence, where customer and staff establish mutual gaze prior to the verbal greetings, thus signaling availability for interaction. The duration of mutual gaze and the timing of the greeting, however, differ between the two varieties. We have also conducted a multi modal analysis of gaze behavior in correlation to the greeting. We found that the customers and staff in the Finland Swedish data share mutual gaze before and during the verbal greeting, and often avert gaze after the verbal greetings. However, in the Sweden Swedish data, the participants often avert gaze before the verbal greetings. Our results thus indicate that both similarities and differences in pragmatic routines and bodily behavior exist between the two national varieties of Swedish. The present study on greeting practices in Finland Swedish and Sweden Swedish should contribute to the field of variational pragmatics and to the development of pluricentric theory.
-
(2018): Virtual Character Facial Expressions Influence Human Brain and Facial EMG Activity in a Decision-Making Game. In: IEEE Transactions on Affective Computing 9 (2), S. 285-298. DOI: 10.1109/TAFFC.2016.2601101
Abstract: We examined the effects of the emotional facial expressions of a virtual character (VC) on human frontal electroencephalographic (EEG) asymmetry (putatively indexing approach/withdrawal motivation), facial electromyographic (EMG) activity (emotional expressions), and social decision making (cooperation/defection). In a within-subjects design, the participants played the Iterated Prisoner's Dilemma game with VCs with different dynamic facial expressions (predefined or dependent on the participant's electrodermal and facial EMG activity). In general, VC facial expressions elicited congruent facial muscle activity. However, both frontal EEG asymmetry and facial EMG activity elicited by an angry VC facial expression varied as a function of preceding interactional events (human collaboration/defection). Pre-decision inner emotional-motivational processes and emotional facial expressions were dissociated, suggesting that human goals influence pre-decision frontal asymmetry, whereas display rules may affect (pre-decision) emotional expressions in human-VC interaction. An angry VC facial expression, high pre-decision corrugator EMG activity, and relatively greater left frontal activation predicted the participant's decision to defect. Both post-decision frontal asymmetry and facial EMG activity were related to reciprocal cooperation. The results suggest that the justifiability of VC emotional expressions and the perceived fairness of VC actions influence human emotional responses.
2017
-
(2017): Real-time gesture translation in intercultural communication. In: AI & Society 32 (1), S. 25-35. DOI: 10.1007/s00146-014-0573-4
Abstract: Nonverbal behavior plays a crucial role in human communication and often leads to misunderstandings between people from different cultures, even if they speak the same language fluently. While translation systems are available for verbal communication, translators for nonverbal communication do not exist yet. We present the conceptual design and an early prototype of a real-time gesture translator using body tracking and gesture recognition in avatar-mediated intercultural interactions. It contributes to the ambitious goal of bridging between cultures by translating culture-specific gestures to enhance mutual understanding. Possible applications of the gesture translator are discussed as a facilitating tool for global business meetings and as a means of technology-enhanced conflict resolution and prevention.
Keywords: body tracking, Computerwissenschaft, Cultural interaction, culture specific, Disziplin, Gestik, Gestik/Mimik, gesture, Interaktion, Kommunikation, kulturspezifisch, Kulturtheorie, Mensch-Technik-Relationen (MTR), Nonverbal behaviour, Nonverbales Verhalten, Personenerkennung, Realtechnik, real-time gesture translation, Sozial Intelligente Agenten, Status / biologische Marker, Technik, translators for nonverbal communication -
(2017): Evaluating the Sensitivity to Virtual Characters Facial Asymmetry in Emotion Synthesis. In: Applied Artificial Intelligence 31 (2), S. 103-118. DOI: 10.1080/08839514.2017.1299983
Abstract: The use of expressive Virtual Characters is an effective complementary means of communication for social networks offering multi-user 3D-chatting environment. In such contexts, the facial expression channel offers a rich medium to translate the ongoing emotions conveyed by the text-based exchanges. However, until recently, only purely symmetric facial expressions have been considered for that purpose. In this article we examine human sensitivity to facial asymmetry in the expression of both basic and complex emotions. The rationale for introducing asymmetry in the display of facial expressions stems from two well-established observations in cognitive neuroscience: first that the expression of basic emotions generally displays a small asymmetry, second that more complex emotions such as ambivalent feeling may reflect in the partial display of different, potentially opposite, emotions on each side of the face. A frequent occurrence of this second case results from the conflict between the truly felt emotion and the one that should be displayed due to social conventions. Our main hypothesis is that a much larger expressive and emotional space can only be automatically synthesized by means of facial asymmetry when modeling emotions with a general Valence-Arousal-Dominance dimensional approach. Besides, we want also to explore the general human sensitivity to the introduction of a small degree of asymmetry into the expression of basic emotions. We conducted an experiment by presenting 64 pairs of static facial expressions, one symmetric and one asymmetric, illustrating eight emotions (three basic and five complex ones) alternatively for a male and a female character. Each emotion was presented four times by swapping the symmetric and asymmetric positions and by mirroring the asymmetrical expression. Participants were asked to grade, on a continuous scale, the correctness of each facial expression with respect to a short definition. Results confirm the potential of introducing facial asymmetry for a subset of the complex emotions. Guidelines are proposed for designers of embodied conversational agent and emotionally reflective avatars.
Keywords: Asymmetrie, Augmented & Virtual Reality, Authentizität von dargestellter Emotion, dominance, Emotionserkennung, Gesichtszüge, Gestik/Mimik, Kognitionswissenschaft/Social Sciences/Humanities, Konventionen, Mensch-Technik-Relationen (MTR), Neurowissenschaften, Realtechnik, soziale Kognition, Technik -
(2017): Towards a social functional account of laughter. Acoustic features convey reward, affiliation, and dominance. In: PLoS one 12 (8). DOI: 10.1371/journal.pone.0183811
DOI: http://www.ncbi.nlm.nih.gov/pubmed/28850589 Abstract: Recent work has identified the physical features of smiles that accomplish three tasks fundamental to human social living: rewarding behavior, establishing and managing affiliative bonds, and negotiating social status. The current work extends the social functional account to laughter. Participants (N = 762) rated the degree to which reward, affiliation, or dominance (between-subjects) was conveyed by 400 laughter samples acquired from a commercial sound effects website. Inclusion of a fourth rating dimension, spontaneity, allowed us to situate the current approach in the context of existing laughter research, which emphasizes the distinction between spontaneous and volitional laughter. We used 11 acoustic properties extracted from the laugh samples to predict participants' ratings. Actor sex moderated, and sometimes even reversed, the relation between acoustics and participants' judgments. Spontaneous laughter appears to serve the reward function in the current framework, as similar acoustic properties guided perceiver judgments of spontaneity and reward: reduced voicing and increased pitch, increased duration for female actors, and increased pitch slope, center of gravity, first formant, and noisiness for male actors. Affiliation ratings diverged from reward in their sex-dependent relationship to intensity and, for females, reduced pitch range and raised second formant. Dominance displayed the most distinct pattern of acoustic predictors, including increased pitch range, reduced second formant in females, and decreased pitch variability in males. We relate the current findings to existing findings on laughter and human and non-human vocalizations, concluding laughter can signal much more that felt or faked amusement.
Keywords: Acoustics, Adult, Aged, Art der Beziehung zum Interaktionspartner, Belohnung, center of gravity, Dauer, Female, formant, Gender, Gestik/Mimik, Humans, Klanghöhe (pitch), Kommunikations-/Interaktionsprinzipien, Lächeln, Lachen, Laughter/psychology, Male, Middle Aged, noisiness, pitch slope, Reward, Sanktionierung, Sex Factors, Social Dominance, Social Identification, Soziale Distanz, Soziale Dominanz, Spontaneität beim Lachen, Status / biologische Marker, Stimmhaftigkeit (voicing), Young Adult, Zugehörigkeit (affiliation) 2016
-
(2016): Perceptions of power and interactional dominance in interpersonal relationships. In: Journal of Social and Personal Relationships 22 (2), S. 207-233. DOI: 10.1177/0265407505050944
Abstract: This investigation uses dyadic power theory (Dunbar, 2000, 2004; Rollins & Bahr, 1976) to examine the relationship between dominance and power and the behavioral manifestations of power in close relationships. Ninety-seven couples (58 married, 39 cohabiting) completed a problem-solving task together while being videotaped. The videotapes were coded for a variety of verbal and nonverbal dominant control attempts including dysfluencies, interruptions, frequency of adaptor and illustrator gestures, vocal characteristics, and general perceptions of dominance. The results revealed that individuals' perceptions of power led to more dominant communication behavior during discussions with their partner. Comparisons between the perceptions of participants and observers and the implications for future research are also discussed.
-
(2016): Reading palm-up signs. Neurosemiotic overview of a common hand gesture. In: Semiotica (210). DOI: 10.1515/sem-2016-0053
Abstract: This article explores ways in which the human nervous system encodes and decodes palm-up gestural signs, signals, and cues. Palm-up gestures and their accompanying speech acts evolved from an ancient neurological system that gave rise to both gestural (pectoral) communication and vocal (laryngeal) language (Bass and Chagnaud 2013, Shared developmental and evolutionary origins for neural basis of vocal-acoustic and pectoral-gestural signaling. Proceedings of the National Academy of Sciences 109. 10677-10684.). Meanings of palm-up cues are multifaceted and nuanced, and express degrees of emotional helplessness, cognitive uncertainty, prosodic emphasis, and social deference. By themselves or in combination with other hand movements - such as reaching, showing, and pointing - palm-up cues are used to begin speaking turns, ask questions, request favors, and share personal opinions, feelings, and moods. The palm-up hand movement is a possibly universal signal of deference, in Erving Goffman's (1956, The nature of deference and demeanor. American Anthropologist 58(3). 473-502.) sense of the term, not unlike other deferential body-motion cues such as the anjaimudra, bow, curtsy, genuflection, kowtow, namaste, poussi-poussi, pranama, sampeah, and wai.
-
(2016): Generating Robot Gaze on the Basis of Participation Roles and Dominance Estimation in Multiparty Interaction. In: ACM Transactions on Interactive Intelligent Systems 5 (4), S. 1-23. DOI: 10.1145/2743028
Abstract: Gaze is an important nonverbal feedback signal in multiparty face-to-face conversations. It is well known that gaze behaviors differ depending on participation role: speaker, addressee, or side participant. In this study, we focus on dominance as another factor that affects gaze. First, we conducted an empirical study and analyzed its results that showed how gaze behaviors are affected by both dominance and participation roles. Then, using speech and gaze information that was statistically significant for distinguishing the more dominant and less dominant person in an empirical study, we established a regression-based model for estimating conversational dominance. On the basis of the model, we implemented a dominance estimation mechanism that processes online speech and head direction data. Then we applied our findings to human-robot interaction. To design robot gaze behaviors, we analyzed gaze transitions with respect to participation roles and dominance and implemented gaze-transition models as robot gaze behavior generation rules. Finally, we evaluated a humanoid robot that has dominance estimation functionality and determines its gaze based on the gaze models, and we found that dominant participants had a better impression of less dominant robot gaze behaviors. This suggests that a robot using our gaze models was preferred to a robot that was simply looking at the speaker. We have demonstrated the importance of considering dominance in human-robot multiparty interaction.
-
(2016): Sozial-Emotionale Kompetenz. In: KINDHEIT UND ENTWICKLUNG 25 (2), S. 114-121. DOI: 10.1026/0942-5403/a000194
Abstract: Recent research has pointed to the importance of the interaction of emotions, perception of emotional expressions in social contexts, and emotion regulation for nonaggressive behavior in children. Evaluations underline the positive impact of prevention programs rather than interventional therapy for children. Long-term curricular training aiming at nonviolent behavior was shown to be successful; however, comprehensive programs cannot always be administered in every-day school contexts. Thus, a short-term prevention program for elementary school fourth-graders was developed for training in the classroom. The program Peaceful Cooperation in Conflict Situations aims at improving the sensitivity and understanding of emotions in self and others, perspective taking in conflict situations, and knowledge of how to communicate nonviolently. In a longitudinal control group study including block randomization, 184 elementary school children in fourth grade participated in the study; 177 complete data records entered the final analysis. The pretest instruments were (a) a paper-and-pencil test measuring the children's social-emotional competencies (i.e., identifying emotional expressions in pictures; describing emotions of aggressors and victims in conflict situations involving children; writing down options of positive behavior to solve the situation peacefully) and (b) teachers' ratings of their pupils' altruistic behavior and emotional and social problems. Two weeks later, the training was administered during three 90-min lessons within a period of 4 weeks. The controls received no special treatment. At posttest 16 weeks after the training, the same tests were administered. Data analyses (MANOVA, ANOVA) revealed that the intervention group improved considerably and with great effects in their social-emotional proficiency as a result of the training. At posttest, they had far advanced in all tested abilities in comparison with their pretest results. They acquired a significantly better understanding of emotions and perspective taking in conflict situations than the controls did. The trained children were rated more favorably by their teachers with respect to their prosocial behavior than at pretest 5 months earlier. The study supports findings from two previous studies with third graders. It shows that a short-term program aiming at enhancing social-emotional knowledge in children can lead to positive results with effects lasting at least several months later. The program can help to prevent the escalation of conflicts and to further peaceful communication. For implementation in schools on a larger scale, measures will have to be taken to ensure the transfer of the reported results.
-
(2016): The Relationship Between Sociomoral Disgust and Physical Disgust. Investigation of Facial Affect in Response to Purity and Fairness Violations. In: NEUROETHICS 9 (2)
Abstract: There has been much recent speculation regarding the relationship between sociomoral disgust, which refers to disgust elicited by moral violations, and the type of disgust elicited by physical stimuli such a rotten food. This study investigated whether there were similar facial expressions of disgust elicited by two types of moral transgressions and physically disgusting behaviors. This study also explored whether facial muscle activity would reflect spontaneous person affect knowledge retrieval after participants were presented with minimal information regarding a person's face and behavior. The two types of moral transgressions were fairness and purity moral transgressions, which are two out of the five moral foundations (purity, fairness, harm, authority, ingroup) in the Moral Foundations Theory (Haidt and Joseph, 2004). Facial muscle activity related to disgust (levator labii) and anger as well as overall negative affect (corrugator supercilii) was recorded while participants associated faces with behavioral statements. The same facial muscle activity was also recorded while participants were shown faces that were previously associated with behavioral statements in a later task. Facial disgust reactions were similar in response to physically disgusting behaviors as well as purity transgressions, but fairness transgressions did not elicit significant facial reactions. Faces that had been previously associated with moral transgressions, physically disgusting, or neutral behaviors also did not evoke facial disgust, showing no transfer of person affective trait knowledge, contrary to previous neuroimaging findings (Todorov et al., 2007). These results suggest that purity vs. fairness transgressions differentially elicit facial disgust reactions, with only purity transgressions eliciting facial disgust activity. These results suggest that these two domains of moral transgressions differ in their similarity to the processing of physical disgust stimuli, consistent with theoretical views that posit that only some moral violations have a basis in the basic emotion of disgust.
Keywords: Bewertung von sozialem Verhalten (/sozialer Angemessenheit), corrugator supercilii, Ekel, Emotionaler Zustand, Fairness, Gesicht und assoziierte Zuschreibungen, Gesichtsausdruck, Gestik/Mimik, levator labii, Mimik, Moral foundations theory by Haidt and Joseph, purity transgressing, (Un)fairness, Wut 2015
-
(2015): The Cultural Significance of a Smile. In: YOUNG 23 (2), S. 154-170. DOI: 10.1177/1103308815569388
Abstract: This article explores meaning-making in Norwegian youth sport. More specifically, how sport media becomes a tool in the instruction of youth athletes. Throughout the 2011-12 season I observed how coaches used portrayals of elite women handballers to educate their 13-year-old girl athletes. Depictions of smiling women were used in contextual attempts to convey the pragmatic benefits of smiling girls. I apply cultural sociology to investigate the processes where coaches mobilize and mould media depictions to inform youth's sport praxis. The account shows why and how the smile's pragmatics is shaped by and shape broad conceptions of youth, sport and gender. Analysis reveals that the smile encourages what the coaches perceive as appropriate strategies towards success. The cultural significance of the smile resides within its capacities to better the performance, to resolve tensions in team hierarchies and allow appropriate displays of aggression in girl handball.
-
(2015): COURTEOUS GESTURE AND DISCOURTEOUS CONTRAST IN FLAMENCA (OCCITAN NOVEL OF XIII CENTURY). In: HIPERBOREEA 2 (1), S. 73-85
Abstract: Coagulated at the end of the 11th century at the seigniorial courts from the southern France and spread afterward in the northern French territory, the courteous ethical code intended to be a set of rules representative for the nobility. It included, between other elegant specifics, the discipline of the body, necessarily based on the self-control and, in the same time, on a distinguished, elegant, polite and moderated gesture. The Middle Ages imaginary and mainly the literature from the 12th and the 13th centuries are full of this kind of examples. In fact, the elite's will to adhere on this ethics and to appropriate it till transforming it in its second nature represents the theme of the courteous novels, focused on some characters which are symbols for the ideal courteous values. By contrast, some other characters are not only at the limit of the courteous model, but also transgress its principles by more or less serious discourteous deviations. The Occitan novel Flamenca (from the 13th century), that we are focusing on, illustrates this trend: one of its principal characters, the signor Archambaut, shows, immediately after his marriage with Flamenca, an unexpected regression, in fact a profound alteration of the courtly behavior, in terms of verbal and mainly gestural manners.
2014
-
(2014): The taxicab-hailing encounter. The politics of gesture in the interaction order. In: Semiotica (202), S. 609-629. DOI: 10.1515/sem-2014-0047
Abstract: The forms of emblematic gesture and other communicative moves used in hailing taxicabs are shaped by the urban built environment and by the micropolitics sparked by what Goffman called the "loose coupling" of the interaction and social orders. Cabdrivers evaluate street hails and choose whether or not to stop for prospective passengers; hailers seek to communicate their desirability as passengers within the limited mode of the street hail. The conditions and constraints that underlie the practice and reception of street hailing are considered. A survey on methods used to hail taxis was sent to a listserv of taxi drivers in San Francisco, California. The drivers evaluated street hails in terms of two axes clarity and propriety - and three levels of context: the immediate interactional context of the street; the anticipated interactional context of the cab ride; and the political context, or the political discourse framing taxicab regulation in the city.
-
(2014): The gentlemen's book of etiquette and manual of politeness. London: Hesperus Classics
Abstract: From advice on how to treat ladies, how to behave at a party, and rules for traveling abroad, this handy 19th-century guidebook will provide entertaining nuggets of wisdom. Ever wondered what to do in polite society if you find an insect in your food? Or how a gentleman should ask a lady to dance? And what on earth is the etiquette for smoking cigars? First published in 1860, this classic guide to gentlemanly behavior is a veritable mine of information and indispensable advice for aspiring gentlemen. No matter if a man finds himself at home, in the street, or in a place of amusement, he has but to leaf through this book to learn how best to behave, and indeed how not to behave. And if he can find time between his numerous invitations to balls and hunting parties, he could benefit from a perusal of the sections on gentlemanly deportment and conversational technique. Not forgetting, of course, the all-important advice on how to treat ladies, surely an integral part of any true gentleman's training.
Keywords: Etiquette for men, Gestik/Mimik, Good manners, Historiographie, History of manners, Kultur, manners, REFERENCE, Regelwerke, Rollen, Selbstdisziplin/-kontrolle, Situationsbedingter Kontext, Soziale Angemessenheit, Soziale Klasse / Schicht, Sozialer Raum / Kultureller Kontext, Sprachgebrauch, Umgangsformen, Werte -
(2014): A Framework for User-Defined Body Gestures to Control a Humanoid Robot. In: International Journal of Social Robotics 6 (3), S. 383-396. DOI: 10.1007/s12369-014-0233-3
Abstract: This paper presents a framework that allows users to interact with and navigate a humanoid robot using body gestures. The first part of the paper describes a study to define intuitive gestures for eleven navigational commands based on analyzing 385 gestures performed by 35 participants. From the study results, we present a taxonomy of the user-defined gesture sets, agreement scores for the gesture sets, and time performances of the gesture motions. The second part of the paper presents a full body interaction system for recognizing the user-defined gestures. We evaluate the system by recruiting 22 participants to test for the accuracy of the proposed system. The results show that most of the defined gestures can be successfully recognized with a precision between 86100 % and an accuracy between 7396 %. We discuss the limitations of the system and present future work improvements.
-
(2014): Subclinical Bias, Manners, and Moral Harm. In: Hypatia 29 (2), S. 287-302. DOI: 10.1111/hypa.12026
Abstract: Mundane and often subtle forms of bias generate harms that can be fruitfully understood as akin to the harms evident in rudeness. Although subclinical expressions of bias are not mere rudeness, like rudeness they often manifest through the breach of mannerly norms for social cooperation and collaboration. At a basic level, the perceived harm of mundane forms of bias often has much to do with feeling oneself unjustly or arbitrarily cut out of a group, a group that cooperates and collaborates but does not do so with me. Appealing to the subtle but familiar choreography of mannered social interaction, I argue, makes it easier to recognize how exclusion can be accomplished through slight but symbolically significant gestures and styles of interaction, where bias manifests not in announced hostility but in an absence of the cooperation and collaboration upon which we rely socially.
2013
-
(2013): A cross-cultural, multimodal, affective corpus for gesture expressivity analysis. In: Journal on Multimodal User Interfaces 7 (1-2), S. 121-134. DOI: 10.1007/s12193-012-0112-x
Abstract: A multimodal, cross-cultural corpus of affective behavior is presented in this research work. The corpus construction process, including issues related to the design and implementation of an experiment, is discussed along with resulting acoustic prosody, facial expressions and gesture expressivity features. However, research work presented here focuses more on the cross-cultural aspect of gestural behavior defining a common corpus construction protocol aiming to identify cultural patterns within non-verbal behavior across cultures i.e. German, Greek and Italian. Culture specific findings regarding gesture expressivity are derived from the affective analysis performed. Additionally, the multimodal aspect, including prosody and facial expressions, is researched in terms of fusion techniques. Finally, a release plan of the corpus to the public domain is discussed aiming to establish the current corpus as a benchmark multimodal, cross-cultural standard and reference point.
2012
-
(2012): Natural interaction with culturally adaptive virtual characters. In: Journal on Multimodal User Interfaces 6 (1-2), S. 39-47. DOI: 10.1007/s12193-011-0087-z
Abstract: Recently, the verbal and non-verbal behavior of virtual characters has become more and more sophisticated due to advances in behavior planning and rendering. Nevertheless, the appearance and behavior of these characters is in most cases based on the cultural background of their designers. Especially in combination with new natural interaction interfaces, there is the risk that characters developed for a particular culture might not find acceptance when being presented to another culture. A few attempts have been made to create characters that reflect a particular cultural background. However, interaction with these characters still remains an awkward experience in particular when it comes to non-verbal interaction. In many cases, human users either have to choose actions from a menu their avatar has to execute or they have to struggle with obtrusive interaction devices. In contrast, our paper combines an approach to the generation of culture-specific behaviors with full body avatar control based on the Kinect sensor. A first study revealed that users are able to easily control an avatar through their body movements and immediately adapt its behavior to the cultural background of the agents they interact with.
-
(2012): “Cars have their own faces”. Cross-cultural ratings of car shapes in biological (stereotypical) terms. In: Evolution and Human Behavior 33 (2), S. 109-120. DOI: 10.1016/j.evolhumbehav.2011.06.003
Abstract: It was recently shown that Austrians associate car front geometry with traits in a way that could be related to face shape geometry mapping to those same overall suites of traits. Yet, possible confounding effects of familiarity with the car models, media coverage and entertainment could not be ruled out. In order to address this, the current study uses a cross-cultural comparison. Adult subjects in two countries (Austria and Ethiopia, n=129) were asked to rate person characteristics of 46 standardized front views of automobiles on various trait scales. These two countries differ substantially with regard to their experience with car models and brands, as well as car marketing and media coverage. Geometric morphometrics was then used to assess the shape information underlying trait attribution. Car shapes for perceived maturity, maleness and dominance were highly similar in both countries, with patterns comparable to shape changes during facial growth in humans: Relative sizes of the forehead and windshield decrease with age/growth, eyes and headlights both become more slit-like, noses and grilles bigger, lips and air-intakes are wider. Austrian participants further attributed various degrees of some interpersonal attitudes and emotions, whereas neither Austrians nor Ethiopians congruently ascribed personalities. Morphological correlates of personal characteristics are discussed, as are person perception and its overgeneralization to inanimate objects. Cross-cultural similarities and differences are addressed, as well as implications for car styling, follow-up studies on driving and pedestrian behavior, and fundamental dimensions in inference from (human) faces.
2011
-
(2011) : Anthropomorphism And Social Robots: Setting Etiquette Expectations In: Hayes, Caroline C.; Miller, Christopher Allan (Hg.): Human-computer etiquette: Cultural expectations and the design implications they place on computers and technology: Boca Raton, Fla.: CRC Press, S. 231-262
Abstract: [-]
Keywords: Anthropomorphisierung, doppelter Treffer, Emotionen, Erwartungen, Etikette, Etiquette as rules of expectation, Favoriten, frames, Gestik/Mimik, Konversationskonventionen als Etikette-Erwartungen, Mensch-Technik-Relationen (MTR), Mensch-Technische Systeme, Modelle/Theorien, Observablen/Kriterien für sozial angemessenes Verhalten und dessen Bewertung, Ohne Abstract oder Text, Realtechnik, Roboter, Rollen, Situationsbedingter Kontext, Sozial Intelligente Agenten, Soziale Angemessenheit, Soziale Robotik, Sozialer Raum / Kultureller Kontext, Soziosensitive Systeme, Status / biologische Marker, Technik 2010
-
(2010): Mobilizing Response. In: Research on Language & Social Interaction 43 (1), S. 3-31. DOI: 10.1080/08351810903471258
Abstract: A fundamental puzzle in the organization of social interaction concerns how one individual elicits a response from another. This article asks what it is about some sequentially initial turns that reliably mobilizes a coparticipant to respond and under what circumstances individuals are accountable for producing a response. Whereas a linguistic approach suggests that this is what oquestionso (more generally) and interrogativity (more narrowly) are for, a sociological approach to social interaction suggests that the social action a person is implementing mobilizes a recipient's response. We find that although both theories have merit, neither adequately solves the puzzle. We argue instead that different actions mobilize response to different degrees. Speakers then design their turns to perform actions, and with particular response-mobilizing features of turn-design speakers can hold recipients more accountable for responding or not. This model of response relevance allows sequential position, action, and turn design to each contribute to response relevance.
2009
-
(2009): The Design of an Expressive Humanlike Socially Assistive Robot. In: Journal of Mechanisms and Robotics 1 (1). DOI: 10.1115/1.2959097
Abstract: It is anticipated that the use of assistive robots will be one of the most important service applications of robotic systems of the future. In this paper, the development of a unique noncontact socially assistive robot consisting of a humanlike demeanor is presented for utilization in hospital wards and nursing/veteran homes to study its role and impact on the well-being of patients, addressing patient's needs and its overall effect on the quality of patient care. The robot will be an embodied entity that will participate in hands-off noncontact social interaction with a patient during the convalescence, rehabilitation, or end-of-life care stage. The robot has been designed as a platform to incorporate the three design parameters of embodiment, emotion, and nonverbal communication to encourage natural human-robot interactions. Herein, we present the overall mechanical design of the socially assistive robot focusing mainly on the development of the actuation system of the face, head, and upper body. In particular, we propose the development of a unique muscle actuation mechanism for the robotic face to allow for the display of rich facial expressions during social assistive interaction scenarios. The novelty of the actuation system is in its use of the dependency of facial muscle activity to minimize the number of individual actuators required to control the robotic face.
Keywords: Design, Disziplin, Embodiment, emotion, Gesicht und assoziierte Zuschreibungen, Gesichtsausdruck, Gestik/Mimik, Ingenieurswissenschaft, Interaktionspartner, Mensch oder Maschine, Mensch-Technik-Relationen (MTR), Nonverbal communication, Nonverbale Kommunikation, Realtechnik, Serviceroboter, Simulation von Emotionen im Gesichtsausdruck, Soziale Robotik, Technik, Übermitteln von Gefühlen/Emotionen -
(2009): Cross-cultural variation of speech-accompanying gesture. A review. In: Language and Cognitive Processes 24 (2), S. 145-167. DOI: 10.1080/01690960802586188
Abstract: This article reviews the literature on cross-cultural variation of gestures. Four factors governing the variation were identified. The first factor is the culture-specific convention for form-meaning associations. This factor is involved in well-known cross-cultural differences in emblem gestures (e.g., the OK-sign), as well as pointing gestures. The second factor is culture-specific spatial cognition. Representational gestures (i.e., iconic and deictic gestures) that express spatial contents or metaphorically express temporal concepts differ across cultures, reflecting the cognitive differences in how direction, relative location and different axes in space are conceptualised and processed. The third factor is linguistic differences. Languages have different lexical and syntactic resources to express spatial information. This linguistic difference is reflected in how gestures express spatial information. The fourth factor is culture-specific gestural pragmatics, namely the principles under which gesture is used in communication. The culture-specificity in politeness of gesture use, the role of nodding in conversation, and the use of gesture space are discussed.
Keywords: culture specificity, Deictische Gesten, Emblem-Gesten, Gesten, Gestik/Mimik, Ikonische Gesten, Kultur, Kulturspezifische Gestik, kulturspezifische Konventionen zur Konzeptualisierung von Ortsangaben, Distanzen etc., Kulturspezische Angemessenheitswahrnehmung von Nicken in einer Konversation, Nicken, OK-Zeichen, Psychologie, Status / biologische Marker, Überblick, Übersicht, Zeigegesten