Does nature create wheels? After intense thinking, I couldn’t find any. In nature, there are bug swarms which can be simulated by having a group of little robots that move by vibrating—the topic of my research internship at University of Waterloo with Professor Gennaro Notomista. This then raises the question of control systems. While there are existing computer vision algorithms to move swarm robots, no one at the time has invented a haptic hand system that mapped onto swarms. Our research involved independently learning advanced linear algebra, multivariable calculus; Figuring out from scratch and software engineering stack to create such a control system for swarms. I learned that you can get less mistakes by using Linux, picked up on using ROS for robotics; Then we used PyTorch to build a neural network to translate what I’ve long understood theoretically before into reality. The data came from a haptic hand that we had which familiarized me with hardware. I realized the barrier of entry to robotics was mainly equipment. Throughout the entire pipeline was troubleshooting and more troubleshooting without knowing the practical possibility. I gained technical skills and it was a seed for my interest in swarm intelligence.
Research and personal significance:
Work in haptic gesture recognition could allow a more intuitive interaction in robotics, surgery, VR and gaming. Haptics could go hand-in-hand with Brain-Computer Interfaces to allow individuals with physical disabilities to regain independence and perform tasks with ease. Currently, there is very little literature on recognizing the gesture from the data (e.g. position, velocity of the joints) of a haptic glove. Most existing work is in computer vision recognizing hand gestures, or using a non-machine learning algorithm for haptic gesture.
Personal significance: This research experience sparked a lot of curiosity in bio-inspired robotics and I envision building an ecosystem of bio-inspired robots. Robotics is hard to enter due to equipment. Now that I have equipment, I gained more confidence in my software and hardware skills. Before, I had a basic proficiency in python and a conceptual understanding of machine learning algorithms.
During the project, I read online tutorials of how to use PyTorch library in python, discussed multivariable calculus and quaternions’ use in code, learned tutorials to use ROS, the linux subsystem on windows and ubuntu. I also learned a few tricks:
We faced obstacles on a daily basis, mostly with various software errors. We have to not just search for the error but also logically eliminate possible sources of error and fixing it.
Likewise I learned what I could improve on:
Steps: