Page 1 of 14

Transactions on Machine Learning and Artificial Intelligence - Vol. 10, No. 1

Publication Date: February, 25, 2022

DOI:10.14738/tmlai.101.11616. Niijima, S., Sasaki, Y., & Mizoguchi, H. (2022). Real-Robot Friendly Passing Motion Planner for Autonomous Navigation in Crowds.

Transactions on Machine Learning and Artificial Intelligence, 10(1). 27-40.

Services for Science and Education – United Kingdom

Real-Robot Friendly Passing Motion Planner for Autonomous

Navigation in Crowds

Shun Niijima

Department of Mechanical Engineering

Tokyo University of Science

2641 Yamazaki, Noda-Shi, Chiba, 278-8510, Japan

Yoko Sasaki

National Institute of Advanced Industrial Science and Technology,

2-3-26 Aomi, Koto-ku, Tokyo 135-0064, Japan

Hroshi Mizoguchi

Department of Mechanical Engineering

Tokyo University of Science

2641 Yamazaki, Noda-Shi, Chiba, 278-8510, Japan

ABSTRACT

This study proposes a real-robot friendly passing motion planner to be used in

crowds. The proposed method learns to pass pedestrians with smooth acceleration

and deceleration by using passing motion learning. A key feature of the proposed

method that it is trained on a simple crowd simulation with both of dynamic and

stationary pedestrians. The learned passing behavior can be used directly in an

autonomous navigation. Evaluations using the crowd simulations indicate that the

proposed method outperforms the existing ones in terms of success rate, arrival

time, and keeping a certain distance from the pedestrians. The proposed navigation

framework is implemented on a mobile robot and demonstrated its successful

navigation between pedestrians in a science museum.

Keywords: passing motion plan, dynamic environment, mobile robot, deep reinforcement

learning

INTRODUCTION

Mobile robots are required to operate in close proximity to

people, such as in shopping malls or exhibition halls. To

move efficiently, many mobile robots use pre-generated map

information of the environment to calculate a path, avoid

obstacles, and move to its destination.

As shown in Figure 1, there are many pedestrians in a real

environment, and a mobile robot needs to detect and avoid

them with its on-board sensors to run safely. Motion

planners usually make use of established parameterized

approaches, such as the dynamic window approach (DWA) Figure 1. Mobile robot in crowds.

Page 2 of 14

28

Transactions on Machine Learning and Artificial Intelligence (TMLAI) Vol 10, Issue 1, February - 2022

Services for Science and Education – United Kingdom

or elastic bands [1], [2], which assume that the current position of the obstacle is static.

However, in reality the obstacles can be dynamic as well. When using these methods, the robot

assumes that the pedestrians are stationary, hence, it may get too close to them or even collide

with them.

Velocity obstacle (VO) [3] was proposed to consider dynamic obstacles. VO is a planner that

generates avoidance maneuvers by selecting the robot velocities outside the collision cone,

which consists of velocities that would result in collision with obstacles moving at given

velocities sometime in the future. The authors of [4] proposed reciprocal velocity obstacle

(RVO), which is an extension of VO in a multi-agent environment. In RVO, each agent moves by

considering the behavior of other agents to achieve a mutual collision avoidance. Moreover, the

authors of [5] used a variant of RVO, namely, the optimal reciprocal collision avoidance (ORCA)

to pro-actively avoid pedestrians. These behavioral policies enable the robot to move efficiently

even among dynamic pedestrians. However, while these policies are sufficient in an artificial

environment where all agents perform uniform movements, they are less optimal in the real

world because pedestrians do not always move as expected. In addition, the motion model of

the robot is not considered, thus, the robot may accelerate and turn rapidly and is forced to run

dangerously.

There has been numerous research on methods for motion planning in dynamic environments.

Several methods have improved performance by incorporating pedestrian motion models into

navigation [6-8]. In [9], a socially aware recurrent neural network model is used to predict

trajectories and set feasible sample waypoints as the optimal path. In [10], a Gaussian process

(GP) is used to learn the motion patterns of pedestrians, and a path planning is performed by

incorporating probabilities into the cost function of random rapid tree search. Although these

methods are effective to ensure that the robot keeps an appropriate distance from the

pedestrians, the freezing robot problem (FRP) [11] may occur when faced with dynamic

pedestrians. For example, because the pedestrian prediction model contains uncertainty, the

robot may not be able to find a safe path through the crowd.

This study proposes a learning method for the passing motion based on deep reinforcement

learning (DRL). An approach of end-to-end motion planner based on DRL is very effective in

difficult to model environments, such as crowds with many pedestrians. The proposed method

trains a passing motion using a simple pedestrian passing simulation. The learned passing

motion planner can be directly integrated into the navigation of a mobile robot, and

experimental results have been obtained on a real robot.

RELATED WORK

With the rapid developments in the machine learning field and DRL, researchers have recently

started using neural networks for robot navigation in dynamic environments [12]. These end- to-end navigation learning methods depend on the environment of the training data, hence, it

is difficult to use them outside the trained environment. Therefore, they are combined with

path planning methods, such as A*, to learn only the motion generation [13,14].

The method described in [15] includes a distributed multi-agent collision avoidance algorithm

based on deep reinforcement learning. This research was later extended by modifying the

reward function to ensure that the agents adhere to social norms [16]. Because these models

Page 3 of 14

29

Niijima, S., Sasaki, Y., & Mizoguchi, H. (2022). Real-Robot Friendly Passing Motion Planner for Autonomous Navigation in Crowds. Transactions on

Machine Learning and Artificial Intelligence, 10(1). 27-40.

URL: http://dx.doi.org/10.14738/tmlai.101.11616.

focus on moving pedestrians, they suffer from stoppages and collisions when faced with

unrecognized pedestrians or stationary obstacles. In a real environment, these problems need

to be addressed because some people stand still for various reasons, for example, to look at

exhibits. Therefore, the aim of this research is to add stationary pedestrians to the learning

process to ensure that the system learns to behave effectively in real environments.

In many cases, the network output is a rough discretized value, such as linear velocity or

angular direction at a certain time. If the obtained behaviors are directly applied to the velocity

commands of the robot, sudden robot velocity and pose changes may occur. Such sudden

changes in speed makes pedestrians uncomfortable, therefore are not suitable for robots that

operate among them. The proposed passing motion learning system learns to accelerate and

decelerate smoothly from the current speed as the robot navigates. In addition, it uses a grid

map of the robot's surroundings as its current state and outputs speed control commands

directly from its input. This eliminates the need for dedicated nodes to calculate velocity

commands and can be easily replaced with existing motion plans.

The main contribution of this study is that it proposes a transit motion learning method that

can be directly integrated into the autonomous navigation of real robots. The main features of

the proposed method are following: 1) the separation of the passage motion from the path

planning makes the learning motion effective in various environments. 2) the method learns

not only from moving pedestrians but also from stationary pedestrians. 3) the passage motion

planning can learn smooth acceleration and deceleration motions to calculate the velocity

command outputs. In the demonstration experiment at the exhibition, the effectiveness of the

proposed method is confirmed by implementing it on a real robot.

PASSING MOTION LEARNING FOR MOBILE ROBOT NAVIGATION

Simulation environment with pedestrians passing by

The learning of passing motion is carried out using a simulation that represents the crowds

with pedestrians and robot. To simulate the passing motion of a robot and a pedestrian, it is

required that pedestrian and robot move purposefully in a limited space. The simulation model

represents pedestrians with a simple circle model. To produce a real-like crowd, the proposed

simulation includes stationary pedestrians as well.

Figure 2 (a) shows the representation of a pedestrian model in simulation, where the position

of the pedestrian is represented as a green circle with a radius �! . The pedestrian circle is

inflated by the radius of the robot �", which denotes the collision area. The predicted range of

pedestrian movement is drawn in the direction of the pedestrian velocity as a cone with a

central angle of 30 degrees. To learn moving without disturbing pedestrians, the simulation

includes the predicted area. The predicted area representation of the 30 degrees cone is

constructed by referring to the forward angle of the pedestrian field of view in the pedestrian

modeling described in [17]. The size of prediction area is scaled using the pedestrian velocity.

The color of the predictive area is represented by a grayscale with a 2 second range at every 0.5

second.

The simulation considers two environments according to the placement of pedestrians. One is

an unobstructed space (figure 2 (b)), while the other contains randomly placed stationary

pedestrians (figure 2 (c)). Walking pedestrians move toward the opposite side from where they