Visual Attention Model for Manipulating Human Attention by a Robot (ICRA2014)

538 Views

March 16, 22

スライド概要

Yusuke Tamura, Shiro Yano, Hisashi Osumi, "Visual Attention Model for Manipulating Human Attention by a Robot," Proceedings of the 2014 IEEE International Conference on Robotics and Automation, pp.5307-5312, 2014.

profile-image

東北大学大学院工学研究科ロボティクス専攻 田村研究室

シェア

埋め込む »CMSなどでJSが使えない場合

関連スライド

各ページのテキスト
1.

Visual Attention Model for Manipulating Human Attention by a Robot Yusuke TAMURA1, Shiro YANO2, Hisashi OSUMI1 1 Chuo University, JAPAN 2 Ritsumeikan University, JAPAN 2014 IEEE International Conference on Robotics & Automation, Hong Kong, June 4, 2014

2.

MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 1. Observation 3. Decision 2

3.

MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision 2

4.

MOTIVATION Smooth interaction between humans... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision Manipulation of other’s attention is important for smooth interaction 2

5.

MOTIVATION Smooth interaction between robot and human... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision 3

6.

MOTIVATION Smooth interaction between robot and human... 2. Estimation/Prediction 0. Elicitation 1. Observation 3. Decision A robot has to understand how its behaviors affect human’s attention. 3

7.

OBJECTIVE 4

8.

OBJECTIVE 4

9.

OBJECTIVE Development of a model of human attention for manipulating human attention by a robot (robot’s) behavior attention area environment 4

10.

RELATED WORKS Computational visual attention model - Saliency map (Itti et al., 1998) - Saliency map + Face position (Cerf et al., 2008) - Saliency map + Face position/orientation (Ozeki et al., 2011) Input image Intensity Colors Orientation Motion Center-surround differences & normalization Across-scale combinations & normalization Saliency map 5

11.

MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. 6

12.

MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. Magic requires a method to achieve an effect. How the trick works What the spectator perceives 6

13.

MANIPULATION OF ATTENTION Magicians are experts of manipulating human attention. Magic requires a method to achieve an effect. How the trick works What the spectator perceives Magicians manipulate spectators’ attention in order to prevent them from uncovering the methods by various skills. Eye gaze (Kuhn et al., 2009) Hand movement (Otero-Millan et al., 2011) Face, hands, and gaze (Akashi et al., 2013) 6

14.

MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. 7

15.

MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map 7

16.

MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map Manipulation map 7

17.

MODEL OF VISUAL ATTENTION Hypothesis: Human visual attention is influenced by visual saliency and others’ face, hands and gaze. Saliency map Manipulation map Attention map 7

18.

SALIENCY MAP Visual saliency of input images Input image Intensity Colors (x2) Orientation (x4) Motion Center-surround differences & normalization Across-scale combinations & normalization Saliency map 8

19.

MANIPULATION MAP Attention area (Akashi et al., 2013) Concentration Distraction Manipulation map describes - Position of face - Position of left/right hand - Direction of gaze 9

20.

FACE & HAND xR Face-Hand map xF F (x) = X i 1 p exp 2⇡ |Si | ⇢ 1 (x 2 xi )T Si 1 (x xi ) xL 10

21.

GAZE x d g xF Gaze map 1 G(x) = p 2⇡ exp d ✓ d(x) 2 d2 2 ◆ 11

22.

MANIPULATION MAP + Face-hand map Gaze map Manipulation map 12

23.

ATTENTION MAP Input&image Face1Hand&map Gaze&map Saliency&map Manipulation&map Attention map Attention&map 13

24.

MAGIC VIDEO A magician performs “vanishing a ball” trick. method: a false transfer of the ball effect: vanish of the ball 1920 x 1080 14

25.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

26.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

27.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

28.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

29.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

30.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

31.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 15

32.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16

33.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16

34.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16

35.

SCENES OF THE VIDEO 1: Raise RH 2: Show the ball in RH 3: Raise LH 4: “Pass” the ball from RH to LH 5: Lower RH and raise LH 6: “Vanish” the ball in LH 8: Open LH 9: Lower LH 7: Change the orientation of LH 16

36.

RESULT: SALIENCY MAP 17

37.

RESULT: SALIENCY MAP 17

38.

RESULT: MANIPULATION MAP 18

39.

RESULT: MANIPULATION MAP 18

40.

RESULT: PROPOSED MODEL 19

41.

RESULT: PROPOSED MODEL 19

42.

MEASUREMENT OF GAZE Measuring participants’ gaze points while watching the magic video 23-­‐inch  screen Participants: 5 healthy volunteers (21-24 yrs) Measurement: faceLAB4.2 (60Hz) cameras (FCB-­‐EX480B,  SONY) 20

43.

EVALUATION INDEX gaze%point:%xg attention%value:% Attention mapA(xg ) evaluation%index:% A(xg ) A(x) saliency%value:% Saliency mapS(xg ) evaluation%index:% Attention'map attention value: Saliency'map saliency value: evaluation index: evaluation index: S(xg ) S(x) 21

44.

RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22

45.

RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22

46.

RESULTS Saliency map Proposed model Gaze point Scene 1 2 3 4 22

47.

RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23

48.

RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23

49.

RESULTS Saliency map Proposed model Gaze point Scene 5 6 7 8 23

50.

Ratio to overall average EVALUATION 5 ** 4 3 2 1 0 Proposed Saliency ** p<0.01 The proposed model can better explain human visual attention than the original saliency map. 24

51.

SUMMARY A model of human attention considering visual saliency and human face, hands and gaze was developed. The model can well explain human attention. FUTURE WORKS Developing an inverse model to calculate appropriate behaviors for any intended attention area. Implementing the model into a small humanoid robot. 25