HRI application demo
Short version demo video with voiceover will appear at HRI 2014 Video Demon Session:
Longer version of the demo video without voiceover:
Video description:
At the beginning of each interaction, the robot blinks its beacon. This signals to the instructor to start the interaction session. The instructor then shows the robot a hand gesture, one out of a set of six (finger counts). Afterwards, the robot blinks its LEDs with a color encoding the predicted class (one to six, where six is a ``fist'' gesture). The instructor then provides the robot with feedback of right or wrong using two special gestures. He hides the glove to indicate a correct prediction, or he waves his fist vigorously to indicate an incorrect prediction.
If the robot's prediction was correct, it goes on to execute its task - in this case to go to the pole with the same color as the LED. Otherwise, it blinks its beacon again to signal the instructor to start a new session (i.e., to repeat the previous gesture).
The Footbot uses its onboard camera to capture an image of the hand gesture. It performs preprocessing on the image, i.e., color-based segmentation and normalization, etc. It then extracts features using a pretrained Convolutional Neural Network. The feature vector resulting from each interaction is then used to predict the gesture class, which also encodes the task that is requested of the robot. After the robot gets the binary feedback, it updates its online multiclass classifier using Upper Confidence Weighted Learning. In this demo, the classifiers are pretrained from only 30 training examples (roughly 5 per class).
Related publication:
A. Giusti, J. Nagi, L. Gambardella, G. A. Di Caro, Cooperative Sensing and Recognition by a Swarm of Mobile Robots, in Proc. of the 25th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vilamoura, Portugal, Oct. 7-12, 2012, pp. 551-558.
Related publications:
A. Giusti, J. Nagi, L. Gambardella, G. A. Di Caro, Distributed Consensus for Interaction between Humans and Mobile Robot Swarms, in Proc. of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS) (Demonstration Track), Valencia, Spain, Jun. 4-8, 2012, pp. 1503-1504.
A. Giusti, J. Nagi, L. Gambardella, S. Bonardi, G. A. Di Caro, Human-Swarm Interaction through Distributed Cooperative Gesture Recognition, in Proc. of the 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI) (Video Session), Boston, USA, Mar. 5-8, 2012, pp. 401-402.
Last modified January 30, 2014.