Human–computer interaction and multimodal user interfaces have evolved dramatically over the last few years offering novel ways of interaction for serious games and virtual environments. Multimodal user interfaces have progressed both in terms of hardware (i.e. basic I/O controls) to sophisticated sensor devices (i.e. body tracking, physiological sensors, etc.) as well as in terms of software from simple graphical user interfaces to advanced user interfaces. This development of new multimodal interaction paradigms has augmented the way serious games and virtual environments are utilized, expanding rapidly the breadth of opportunities for training and simulation. As a result, the user-base has been significantly expanded, and making them accessible to a diverse audience.

There are many factors affecting the evolution of advanced multimodal interaction techniques and user interfaces, such as the broad variety of facets of virtual worlds and computer games that include computer graphics, software engineering, and human–computer interaction techniques. Among these factors can be identified the technology-driven nature of new developments, which often does not take usability and applicability into account, as well as the convergence between new technologies such as: learning interfaces, brain-computer interfaces, exergames, interactive physics simulations and AI techniques, that require multimodal approaches.

For this special issue we have accepted 5 articles that demonstrate the breadth of techniques for advanced multimodal interaction techniques and user interfaces for serious games and virtual environments. These articles are a mix of original submissions and of extended and revised versions of selected best technical papers presented at the 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games 2017), 6–8 September 2017, Athens, Greece.

The first article, entitled "fNIRS-based classification of mind-wandering with personalized window selection for multimodal learning interfaces" by Liu et al., presents a study using functional near-infrared spectroscopy (fNIRS) and investigated machine learning classifiers to detect mind-wandering episodes based on fNIRS data, both on an individual level and a group level, specifically focusing on automated window selection to improve classification results. Results show that the proposed algorithm achieved significant improvement compared to the previous state of the art in terms of brain-based classification of mind-wandering building a foundation for both the evaluation of multimodal learning interfaces and for future attention-aware systems.

The second article, entitled "A BCI video game using neurofeedback improves the attention of children with autism" by Mercado et al., describes the results of a 10-week deployment study with 26 children with severe autism using FarmerKeeper as a tool to support the neurofeedback therapies of children with autism. Pre- and post-assessment evaluation indicated that all children with autism improved their attention, attentional control and sustained attention. Directions for future work and the potential benefits of this new generation of BCI videogames in healthcare scenarios are also discussed.

The third article, entitled "Circus in Motion: a multimodal exergame supporting vestibular therapy for children with autism" by Cibrian et al., presents the design and development of Circus in Motion, a multimodal exergame supporting children with autism with the practice of non-locomotor movements. A controlled experiment with 12 children with autism showed that Circus in Motion excels traditional vestibular therapies in increasing physical activation and the number of movements repetitions. Open challenges and opportunities of multimodal exergames to support motor therapeutic interventions for children with autism in the long-term are finally discussed.

The fourth article, entitled "Neighborhood based decision theoretic rough set under dynamic granulation for BCI motor imagery classification" by Devi and Inbarani, analyses and classifies the brain signals for motor movements to aid in rehabilitation and restoration. By proposing a novel optimization technique with Neighborhood Decision Theoretic Rough Set under Dynamic Granulation. Results indicate that the proposed method Neighborhood based Decision Theoretic Rough Set under Dynamic Granulation gives higher classification accuracy compared to existing approaches.

The final article, entitled "Identifying and evaluating conceptual representations for auditory-enhanced interactive physics simulations" by Tomlinson et al., evaluated sets of audio designs for two different, but contextually- and visually-similar simulations. We identified key aspects of the audio representations and the simulation content which needed to be evaluated were identified. Designs across two simulations were compared to understand which auditory designs could generalize to other simulations. Authors suggest important characteristics to represent through audio for future simulations, provide sound design suggestions, and address how overlap between visual and audio representations can support learning opportunities.

The guest editors wish to thank Prof. Jean-Claude Martin for his guidance in producing this special issue and we also wish to thank the authors and reviewers for their hard work. We hope you enjoy reading these articles and learn more about advanced multimodal interaction techniques and user interfaces for serious games and virtual environments.