A vision based deep learning architecture for automating manufacturing processes with robotics

Project aim

To develop a flexible and intelligent robotic control system that through the use of deep learning, can be controlled by adapted to novel tasks and by humans through high-level planning.

Project background

This project investigates how deep learning can be used to train intelligent robotic control systems to automate the control of machines designed to be operated by humans. It will involve the development of simulations to train deep neural networks to process different types of sensory inputs and to control a robotic arm with a dexterous gripper to carry out manipulation tasks.

After training and development in simulation, the deep neural networks will be transferred to the real world to automate a real laboratory task. Teleoperation with virtual reality will be used to enable humans to operate the robotic arm and see through its camera, both in simulation and the real world. This will allow humans to trial the feasibility of automation tasks to find the most capable arm for the job and gather real human operator data to improve the performance of deep neural networks.