Future human-machine interfaces - 2021-2023

Authors:

  • Britta Levin
  • Zackarias Alenljung
  • Jonas Hermelin
  • Johan O Karlsson
  • Victor Lindholm
  • Agnes Tegen
  • Susanna Nilsson

Publish date: 2024-04-11

Report number: FOI-R--5568--SE

Pages: 43

Written in: Swedish

Keywords:

  • human-machine interface
  • gestures
  • gesture based interaction
  • unmanned ground vehicle
  • UGV
  • motion capture glove
  • AI
  • machine learning
  • ChatGPT
  • language models
  • generative AI
  • XR
  • extended reality
  • MR
  • mixed reality
  • VR
  • virtual reality.

Abstract

The interfaces between human users and technology are constantly evolving, as well as the ways we interact with them. New breakthroughs in AI and machine learning have led to rapid development in a number of research areas. A prominent example is AI assistants, such as Open AI's chatGPT, that has created new possibilities (and challenges) in many different domains, such as business intelligence, decision support and programming. In the display area, technologies such as XR (extended reality) and VR (virtual reality) have become increasingly available both as consumer products and as tools for research. Currently a wide array of functionalities, from gesture and speech input to eye-controlled interaction, are often built directly into the XR interfaces. The project described in this report has monitored this development, which has resulted in survey of technologies and interfaces of relevance to the Swedish Armed Forces. The project has explored gestures as a way of interaction and developed an example of how XR technology can be used in a C2 context. In addition to this, studies have been initialised on how AI, in the form of machine learning, can be used for the analysis and categorization of data from sensors (accelerometers and gyroscopes), as well as part of a concept for how a text-based generative AI assistant can support tactical planning. The report provides an overview of the work conducted within the project over the past three years, which will form the basis for further studies and concept development in the coming years.