Alle Publikationen
- <<
- <
- 1
2019
-
(2019): Understanding of Human Behavior with a Robotic Agent Through Daily Activity Analysis. In: International Journal of Social Robotics 11 (3), S. 437-462. DOI: 10.1007/s12369-019-00513-2
DOI: https://doi.org/10.1007/s12369-019-00513-2 Abstract: Personal assistive robots to be realized in the near future should have the ability to seamlessly coexist with humans in unconstrained environments, with the robot’s capability to understand and interpret the human behavior during human–robot cohabitation significantly contributing towards this end. Still, the understanding of human behavior through a robot is a challenging task as it necessitates a comprehensive representation of the high-level structure of the human’s behavior from the robot’s low-level sensory input. The paper at hand tackles this problem by demonstrating a robotic agent capable of apprehending human daily activities through a method, the Interaction Unit analysis, that enables activities’ decomposition into a sequence of units, each one associated with a behavioral factor. The modelling of human behavior is addressed with a Dynamic Bayesian Network that operates on top of the Interaction Unit, offering quantification of the behavioral factors and the formulation of the human’s behavioral model. In addition, light-weight human action and object manipulation monitoring strategies have been developed, based on RGB-D and laser sensors, tailored for onboard robot operation. As a proof of concept, we used our robot to evaluate the ability of the method to differentiate among the examined human activities, as well as to assess the capability of behavior modeling of people with Mild Cognitive Impairment. Moreover, we deployed our robot in 12 real house environments with real users, showcasing the behavior understanding ability of our method in unconstrained realistic environments. The evaluation process revealed promising performance and demonstrated that human behavior can be automatically modeled through Interaction Unit analysis, directly from robotic agents.
2018
-
(2018): Action Anticipation: Reading the Intentions of Humans and Robots. In: IEEE Robotics and Automation Letters 3 (4), S. 4132-4139. DOI: 10.1109/LRA.2018.2861569
DOI: https://doi.org/10.1109/LRA.2018.2861569 Abstract: Humans have the fascinating capacity of processing nonverbal visual cues to understand and anticipate the actions of other humans. This “intention reading” ability is underpinned by shared motor repertoires and action models, which we use to interpret the intentions of others as if they were our own. We investigate how different cues contribute to the legibility of human actions during interpersonal interactions. Our first contribution is a publicly available dataset with recordings of human body motion and eye gaze, acquired in an experimental scenario with an actor interacting with three subjects. From these data, we conducted a human study to analyze the importance of different nonverbal cues for action perception. As our second contribution, we used motion/gaze recordings to build a computational model describing the interaction between two persons. As a third contribution, we embedded this model in the controller of an iCub humanoid robot and conducted a second human study, in the same scenario with the robot as an actor, to validate the model's “intention reading” capability. Our results show that it is possible to model (nonverbal) signals exchanged by humans during interaction, and how to incorporate such a mechanism in robotic systems with the twin goal of being able to “read” human action intentionsand acting in a way that is legible by humans
Keywords: Blick / Gaze, Body Movement, EYE GAZE, Gaze, Gesichtserkennung, HRI, human-robot interaction, iCub, Intentionserkennung (Roboter erkennt Menschenintention), Körperbewegung, Körperhaltung, Körpersprache, Nichtverbale Kommunikation, nonverbal, Nonverbal behavior, non-verbal cues, Nonverbale Kommunikation, Soziosensitive Systeme 2017
-
(2017): Social Scene Understanding: End-to-End Multi-Person Action Localization and Collective Activity Recognition. Online verfügbar unter http://arxiv.org/pdf/1611.09078v1
Abstract: We present a unified framework for understanding human social behaviors in raw image sequences. Our model jointly detects multiple individuals, infers their social actions, and estimates the collective actions with a single feed-forward pass through a neural network. We propose a single architecture that does not rely on external detection algorithms but rather is trained end-to-end to generate dense proposal maps that are refined via a novel inference scheme. The temporal consistency is handled via a person-level matching Recurrent Neural Network. The complete model takes as input a sequence of frames and outputs detections along with the estimates of individual actions and collective activities. We demonstrate state-of-the-art performance of our algorithm on multiple publicly available benchmarks.
2016
-
(2016): Knowledge Transfer for Scene-specific Motion Prediction. Online verfügbar unter http://arxiv.org/pdf/1603.06987v2
Abstract: When given a single frame of the video, humans can not only interpret the content of the scene, but also they are able to forecast the near future. This ability is mostly driven by their rich prior knowledge about the visual world, both in terms of (i) the dynamics of moving agents, as well as (ii) the semantic of the scene. In this work we exploit the interplay between these two key elements to predict scene-specific motion patterns. First, we extract patch descriptors encoding the probability of moving to the adjacent patches, and the probability of being in that particular patch or changing behavior. Then, we introduce a Dynamic Bayesian Network which exploits this scene specific knowledge for trajectory prediction. Experimental results demonstrate that our method is able to accurately predict trajectories and transfer predictions to a novel scene characterized by similar elements.
-
(2016): Anticipating Human Activities Using Object Affordances for Reactive Robotic Response. In: IEEE transactions on pattern analysis and machine intelligence 38 (1), S. 14-29. DOI: 10.1109/TPAMI.2015.2430335
DOI: http://www.ncbi.nlm.nih.gov/pubmed/26656575 Abstract: An important aspect of human perception is anticipation, which we use extensively in our day-to-day activities when interacting with other humans as well as with our surroundings. Anticipating which activities will a human do next (and how) can enable an assistive robot to plan ahead for reactive responses. Furthermore, anticipation can even improve the detection accuracy of past activities. The challenge, however, is two-fold: We need to capture the rich context for modeling the activities and object affordances, and we need to anticipate the distribution over a large space of future human activities. In this work, we represent each possible future using an anticipatory temporal conditional random field (ATCRF) that models the rich spatial-temporal relations through object affordances. We then consider each ATCRF as a particle and represent the distribution over the potential futures using a set of particles. In extensive evaluation on CAD-120 human activity RGB-D dataset, we first show that anticipation improves the state-of-the-art detection results. We then show that for new subjects (not seen in the training set), we obtain an activity anticipation accuracy (defined as whether one of top three predictions actually happened) of 84.1, 74.4 and 62.2 percent for an anticipation time of 1, 3 and 10 seconds respectively. Finally, we also show a robot using our algorithm for performing a few reactive responses.
Keywords: Affordance, affordances, Affordanz, Algorithms, Aniticipation, Human Activities, Humans, Imaging, Intentionserkennung (Roboter erkennt Menschenintention), J. J. Gibson's theory of affordance, machine learning, MODELS, Movement, PERCEPTION, Psychological, Robotics, Soziosensitive Systeme, Soziosensitivität, Statistical, Three-Dimensional, Video Recording 2015
-
(2015) : Social context perception for mobile robots In: Burgard, Wolfram: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): Sept. 28, 2015 - Oct. 2, 2015, Hamburg, Germany: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS): Hamburg, Germany: 9/28/2015 - 10/2/2015. IEEE/RSJ International Conference on Intelligent Robots and Systems; Iros: Piscataway, NJ: IEEE, S. 3621-3627
Abstract: As robots enter human spaces, unique perception challenges are emerging. Sensing human activity, adapting to highly dynamic environments, and acting coherently and contingently is challenging when robots transition from structured environments to human-centric ones. We approach this problem by employing context-based perception, a biologically-inspired, low-cost approach to sensing that leverages noisy, global features. Across several months, our mobile robot collected real-world, multimodal data from multi-use locations; where the same space might be used for many different activities. We then ran a series of unimodal and multimodal classification experiments. We successfully classified several aspects of situational context from noisy data, and, to our knowledge are the first group to do so. This work represents an important step toward enabling robots that can readily leverage context to solve perceptual tasks.
2011
-
(2011) : Please do not disturb! Minimum interference coverage for social robots In: Staff, IEEE: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems: 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2011): San Francisco, CA: 9/25/2011 - 9/30/2011. IEEE Staff: [Place of publication not identified]: IEEE, S. 1968-1973
Abstract: In this paper we address the problem of human-aware coverage planning. We first present an approach to learn and model human activity events in a probabilistic spatio-temporal map using spatial Poisson processes. We then propose a coverage planner for paths that minimize the interference probability with people. To this end, we pose the coverage problem as an asymmetric traveling salesman problem with time-dependent costs (ATDTSP) derived from the information in the map. The approach enables a noisy robotic vacuum in a home scenario, for instance, to learn to avoid busy places at certain times of the day such as the kitchen at lunch time. We evaluate the planner using a simulator of people in a home environment to generate typical weekday activity patterns. In the experiments with a regular TSP planner and two modified TSP heuristics, the proposed coverage planner significantly reduces interference with people in terms of number of disturbed persons and overall disturbance time.
- <<
- <
- 1