Controlling with gestures - Overview and evaluation of gesture based interaction

Authors:

  • Johan O Karlsson
  • Jonas Hermelin
  • Britta Levin
  • Susanna Nilsson

Publish date: 2023-01-11

Report number: FOI-R--5381--SE

Pages: 39

Written in: Swedish

Keywords:

  • human-machine interface
  • gesture based interaction
  • unmanned ground vehicle
  • UGV
  • machine learning
  • motion capture glove
  • human autonomy teaming

Abstract

The Swedish Armed Forces have long used gestures to communicate orders and intent at short as well as long distances, both within and between groups, or teams. Gesture-based communication has primarily been used for communication between people, but this will likely change when autonomous systems and vehicles, such as UGVs (Unmanned Ground Vehicles), are also included in the teams. The objective of the work described in this report was to create the basis for a proof of concept demonstration of gesture control of a UGV using commercially available technology. The report presents a theoretical background with focus on gestures as a tool for communication and interaction, as well as the results of two initial technical evaluations of a motion capture glove used for interaction with a UGV. The study showed that it is possible to use the motion capture gloves for interaction with a UGV, but that further work is required to determine which gestures are suitable for different scenarios and different types of interaction. Further work is also required to adjust the underlying machine learning model for registration and interpretation of the gestures.