---
title: Robotic Shopping Guidance System for the Visually Impaired Users using Servo Brakes (AIM2024)
tags:  #ショッピングカート  
author: [Tamura Lab.](https://www.docswell.com/user/tamlab)
site: [Docswell](https://www.docswell.com/)
thumbnail: https://bcdn.docswell.com/page/87DYMX257G.jpg?width=480
description: Zhan Shi, Yusuke Tamura, Zhenyu Liao, Weizan He, Yasuhisa Hirata, "Robotic Shopping Guidance System for the Visually Impaired Users using Servo Brakes," Proceedings of the 2024 IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp.1416-1421, 2024.
published: July 19, 24
canonical: https://www.docswell.com/s/tamlab/5N1M7V-AIM2024
---
# Page. 1

![Page Image](https://bcdn.docswell.com/page/87DYMX257G.jpg)

Robotic Shopping Guidance System for
the Visually Impaired Users using Servo Brakes
Zhan SHI, Yusuke TAMURA, Zhenyu LIAO, Weizan HE, Yasuhisa HIRATA
Dept. of Robotics, Tohoku University, Japan
z.shi@srd.mech.tohoku.ac.jp
ytamura@tohoku.ac.jp
2024 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM 2024), Boston, USA | July 18


# Page. 2

![Page Image](https://bcdn.docswell.com/page/VJP6RP1DE8.jpg)

Motivation
285 million people are estimated to be living with visual impairments.
(39 million: blind, 246 million: low vision)
Effects on QoL:
- Reduced independence in daily activities
- Challenges in navigation and mobility
- Difficulties in accessing information
Daily living challenges:
personal care, household management, social interactions, shopping, …
2


# Page. 3

![Page Image](https://bcdn.docswell.com/page/2EV3R2YG7Q.jpg)

Challenges in Supermarket Shopping
Locating Items
- Difficulty in finding specific products within large store
layouts
Navigation Barriers
- Avoiding obstacles and safely navigating crowded aisles
Product Identification
- Challenges in recognizing and distinguishing between
similar products
- Reading price tags and ingredient labels is often
impossible without assistance
3


# Page. 4

![Page Image](https://bcdn.docswell.com/page/57G3MRWDJL.jpg)

Related Studies
Voice navigation
NavCog [Ahmetovic+ 2016]
NaviLens
Cons
Pros
- Detailed instructions - Environmental noise
- Privacy
- Directional cues
- Cognitive load
- Cannot ensure the safety
Autonomous navigation
Co-robotic cane [Ye+ 2016]
AI suitcase (CaBot)
[Guerreiro+ 2019]
Pros
- Safer path to avoid
obstacles
Cons
- Risk of unexpected behavior
- Lack of “sense of control”
4


# Page. 5

![Page Image](https://bcdn.docswell.com/page/4EQ5RV3XJP.jpg)

Objective
Development of a robotic assistant system that safely guides visually
impaired individuals in supermarkets
- Provide appropriate feedback to users
- Ensure user safety
- Maintain user’s sense of control
- Seamlessly integrate into the shopping environment
5


# Page. 6

![Page Image](https://bcdn.docswell.com/page/KJ498M1271.jpg)

Design Strategies
Provide appropriate feedback to users
Tactile/Force feedback are useful for intuitive understanding of surroundings.
Auditory feedback is useful for detailed instructions.
Ensure user safety
It is necessary to recognize the surrounding environment and avoid collisions.
Maintain user’s sense of control
It is desirable for users to move independently.
Seamlessly integrate into the shopping environment
The appearance of the robotic system should be suitable for use in a supermarket.
Shopping cart-type passive robot with servo brakes
6


# Page. 7

![Page Image](https://bcdn.docswell.com/page/LE1K28GKJG.jpg)

Shopping Cart-type Passive Robot
RGBD camera & IMU
RGB-D camera
Controller
Speaker
Speaker
Left brake
Right brake
Servo brakes
Controllers
Battery
7


# Page. 8

![Page Image](https://bcdn.docswell.com/page/GEWDRZKPJ2.jpg)

Brake Control
Stop: Activate both brakes at the same time
Turn Left: Activate the left side brake
Turn Right: Activate the right side brake
8


# Page. 9

![Page Image](https://bcdn.docswell.com/page/47ZGR1Z6E3.jpg)

Navigation
Global path planning
- Represent the environment as a graph with nodes
- Use a graph-based algorithm to calculate the
15
12
9
6
3
0
16
13
10
7
4
1
17
14
11
8
5
2
optimal path
At a node:
The brakes are controlled to face the direction of the
next node using IMU based on the relative position
between the current node and next node.
Between nodes:
When a marker is detected
The brakes are adjusted to move towards the
marker.
9


# Page. 10

![Page Image](https://bcdn.docswell.com/page/YJ6YRLZL7V.jpg)

Marker-based Navigation
Torque (Nm)
Turn Left
Turn Right
12
8
4
0
-35 -30
-5
0
5
30
35
Target direction θ (deg)
ArUco Marker
✓
<latexit sha1_base64="i88lUfDMtsFoMgl+H48iZ0wr3Ao=">AAACaXichVG7SgNBFD1Z3/GRqI1oE1wiVmEivrASbSx95QFJCLvraEY3u8vuJKDBH7CyE7VSEBE/w8YfsPATxDKCjYV3NwuiQb3DzJw5c8+dMzO6YwpPMvYcUTo6u7p7evui/QODQ7H48EjWs2uuwTOGbdpuXtc8bgqLZ6SQJs87Ltequslz+sGqv5+rc9cTtrUtDx1eqmp7ltgVhiaJyhZlhUutHFdZigWRaAfpEKgIY92O36KIHdgwUEMVHBYkYRMaPGoFpMHgEFdCgziXkAj2OY4RJW2NsjhlaMQe0LhHq0LIWrT2a3qB2qBTTOouKRNIsid2x5rskd2zF/bxa61GUMP3ckiz3tJypxw7Gdt6/1dVpVmi8qX607PELhYDr4K8OwHj38Jo6etHZ82tpc1kY4pds1fyf8We2QPdwKq/GTcbfPMSUfqA9M/nbgfZmVR6PjW3Masur4Rf0YsJTGKa3nsBy1jDOjJ07j5OcY6LyKsyrIwp461UJRJqRvEtFPUTrt2MLg==</latexit>
10


# Page. 11

![Page Image](https://bcdn.docswell.com/page/GJ5G81WMJ4.jpg)

Experiments
Environment:
Simplified environment simulating
supermarket’s unique right-angle
turning paths
ID: 0
ID: 1
ID: 3
Participants:
6 healthy young participants with
blindfolds
Procedure:
Participants complete navigation tasks
on two routes
ID: 2
ID: 4
11


# Page. 12

![Page Image](https://bcdn.docswell.com/page/LE3R814975.jpg)

ID: 3
ID: 0
ID: 4
Path: 0 ▶︎ 3 ▶︎ 4
Auditory feedback:
12


# Page. 13

![Page Image](https://bcdn.docswell.com/page/8EDYMXQ8EG.jpg)

Experimental Results
Task completion rate:
12 times out of 12 (100%), participants successfully reached their destination.
Task completion time:
Proposed method (1st trial)
Proposed method (2nd trial)
Baseline (with caregiver)
0
5
10
15
20
25
30
35
Completion time (s)
Subjective evaluation by participants (scale 1-10):
Perceived Safety: 8.2
Ease of Use: 8.0
Movement Smoothness: 9.5 (linear motion), 7.5 (rotation)
13


# Page. 14

![Page Image](https://bcdn.docswell.com/page/V7P6RPLWJ8.jpg)

Conclusions
We developed a robotic assistant system that safely guides visually
impaired individuals in supermarkets.
Shopping cart-type passive robot with servo brakes
Future works
- Navigation in crowded environment
- Integration with product identification
- Field test
Acknowledgment
This work was partially supported by JST COI-NEXT Grant Number JPMJPF2201.
14