Eye Control and Motion with Deep Reinforcement Learning: In Virtual and Physical Environments

Arizmendi, SergioSergioArizmendiPaz, AsdrubalAsdrubalPazGonzález, JavierJavierGonzálezPonce, HiramHiramPonce2024-03-012024-03-012024-01-01Arizmendi, S., Paz, A., González, J., Ponce, H. (2024). Eye Control and Motion with Deep Reinforcement Learning: In Virtual and Physical Environments. In: Calvo, H., Martínez-Villaseñor, L., Ponce, H. (eds) Advances in Computational Intelligence. MICAI 2023. Lecture Notes in Computer Science(), vol 14391. Springer, Cham. https://doi.org/10.1007/978-3-031-47765-2_89783031477645https://scripta.up.edu.mx/handle/20.500.12552/998510.1007/978-3-031-47765-2_82-s2.0-85177227465Attention mechanism in computer vision refers to scan, detect, and track a target object. This paper aims to develop and virtually train a machine learning model for object attention mechanism, combining object detection and mechanical automation. For this, we use Unity 3D Engine to model a simple scene in which two virtual cameras align together to realize a monocular attention in specific objects. Deep reinforcement learning, via ML-agent’s library, was used to train a model that aligns the virtual cameras. Moreover, the model was transferred to a physical camera to replicate the performance of attention mechanism.enMachine learningDeep reinforcement learningM.L. AgentUnityLearning environmentNeural NetworkEye Control and Motion with Deep Reinforcement Learning: In Virtual and Physical EnvironmentsResource Types::text::book::book part