Simulation of Obstacle Avoidance for Multiple Autonomous Vehicles in a Dynamic Environment Using Q-Learning

The availability of inexpensive, yet competent hardware allows for increased level of automation and self-optimization in the context of Industry 4.0. However, such agents require high quality information about their surroundings along with a robust strategy for collision avoidance, as they may cause expensive damage to equipment or other agents otherwise. Manually defining a strategy to cover all possibilities is both time-consuming and counter-productive given the capabilities of modern hardware. This paper explores the idea of a model-free self-optimizing obstacle avoidance strategy for multiple autonomous agents in a simulated dynamic environment using the Q-learning algorithm.

Authors:

• Andreas D. Jansson

Keywords:

• Autonomous vehicles
• industry 4.0
• multi-agent system
• obstacle avoidance
• Q-learning
• simulation.

References:

[1] G. G. Meyer, K. Främling, and J. Holmström, “Intelligent products: a survey”, Computers in Industry, vol. 60, issue 3, pp. 137-148, Apr. 2009.
[2] T. Skjoett-Larsen, “European logistics beyond 2000”, International Journal of Physical Distribution & Logistics Management, vol. 30, issue 5, pp. 377-387, June 2000.
[3] R. Sella, A. Rassõlkinb, R. Wanga, and T. Otto, “Integration of autonomous vehicles and industry 4.0”, in Proceedings of the Estonian Academy of Sciences, Tallin, Estonia, 2019, pp. 389–394.
[4] Y. Cheng, and G. Y. Wang, “Mobile robot navigation based on lidar”, 2018 Chinese Control and Decision Conference (CCDC), Shenyang, China, 2018, pp. 1243-1246.
[5] F. B. P. Malavazi, R. Guyonneau, J.-B Fasquel, S. Lagrange, and F. Mercier, “LiDAR-only based navigation algorithm for an autonomous agricultural robot”, Computers and Electronics in Agriculture, vol. 154, pp. 71-79, Nov. 2018.
[6] B Z. Su, X. Zhou, T. Cheng, H. Zhang, B. Xu, and W. Chen, "Global localization of a mobile robot using lidar and visual features," 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO), Macau, Macao, 2017, pp. 2377-2383.
[7] Z. Yan, L. Sun, T. Duckctr, and N. Bellotto, "Multisensor online transfer learning for 3D LiDAR-based human detection with a mobile robot," 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain, 2018, pp. 7635-7640.
[8] M. Freese, S. Singh., F. Ozaki, and N. Matsuhira, “Virtual robot experimentation platform V-REP: a versatile 3D robot simulator” in Simulation, Modeling, and Programming for Autonomous Robots, N. Ando, S. Balakirsky, T. Hemker, M. Reggiani, and O. von Stryk, Ed. Berlin Heidelberg: Springer-Verlag, 2010, pp. 51-62.
[9] S. Nakaoka, "Choreonoid: Extensible virtual robot environment built on an integrated GUI framework," 2012 IEEE/SICE International Symposium on System Integration (SII), Fukuoka, Japan, 2012, pp. 79-85.
[10] R. S. Sutton, and A. G. Barto, Reinforcement learning: an introduction, 2nd edition. Cambridge, MA: The MIT Press, 2015, ch. 6.5.