Publications
2023 🔗
- Identification of Language-Induced Mental Load from Eye Behaviors in Virtual RealityJournal PaperSensors 2023DOI Website
Experiences of virtual reality (VR) can easily break if the method of evaluating subjective user states is intrusive. Behavioral measures are increasingly used to avoid this problem. One such measure is eye tracking, which recently became more standard in VR and is often used for content-dependent analyses. This research is an endeavor to utilize content-independent eye metrics, such as pupil size and blinks, for identifying mental load in VR users. We generated mental load independently from visuals through auditory stimuli. We also defined and measured a new eye metric, focus offset, which seeks to measure the phenomenon of “staring into the distance” without focusing on a specific surface. In the experiment, VR-experienced participants listened to two native and two foreign language stimuli inside a virtual phone booth. The results show that with increasing mental load, relative pupil size on average increased 0.512 SDs (0.118 mm), with 57% reduced variance. To a lesser extent, mental load led to fewer fixations, less voluntary gazing at distracting content, and a larger focus offset as if looking through surfaces (about 0.343 SDs, 5.10 cm). These results are in agreement with previous studies. Overall, we encourage further research on content-independent eye metrics, and we hope that hardware and algorithms will be developed in the future to further increase tracking stability.
@article{2023-Schirm-LanguageBoothVR, author = {Schirm, Johannes and Gómez-Vargas, Andrés Roberto and Perusquía-Hernández, Monica and Skarbez, Richard T. and Isoyama, Naoya and Uchiyama, Hideaki and Kiyokawa, Kiyoshi}, title = {Identification of Language-Induced Mental Load from Eye Behaviors in Virtual Reality}, journal = {Sensors}, year = {2023}, volume = {23}, number = {15}, issn = {1424-8220}, doi = {10.3390/s23156667}, article-number = {6667}, }
2022 🔗
- Identifying Language-induced Mental Load from Eye Behaviors in Virtual RealityPosterIn ICAT-EGVE 2022 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments - Posters and Demos 2022DOI
We compared content-independent eye tracking metrics under different levels of language-induced mental load in virtual reality (VR). We designed a virtual environment to balance consistent recording of eye data with user experience and freedom of action. We also took steps towards quantifying the phenomenon of not focusing exactly on surfaces by proposing "focus offset" as a VR-compatible eye metric. Responses to conditions with higher mental load included larger and more variable pupil sizes and less fixations. We also observed less voluntary gazing at distraction content, and a tendency of looking through surfaces.
@inproceedings{2022-Schirm-LanguageBoothVR, author = {Schirm, Johannes and Perusqu\'{\i}a-Hern\'{a}ndez, Monica and Isoyama, Naoya and Uchiyama, Hideaki and Kiyokawa, Kiyoshi}, title = {Identifying Language-induced Mental Load from Eye Behaviors in Virtual Reality}, booktitle = {ICAT-EGVE 2022 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments - Posters and Demos}, year = {2022}, editor = {Teo, Theophilus and Kondo, Ryota}, publisher = {The Eurographics Association}, isbn = {978-3-03868-192-2}, doi = {10.2312/egve.20221290}, issn = {1727-530X}, }
2021 🔗
- Compelling AR Earthquake Simulation with AR Screen ShakingConference PaperIn ICAT-EGVE 2021 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments 2021DOI
Many countries around the world suffer losses from earthquake disasters. To reduce the injury of individuals, safety training is essential to raise people’s preparedness. To conduct virtual training, previous work uses virtual reality (VR) to mimic the real world, without considering augmented reality (AR). Our goal is to simulate earthquakes in a familiar environment, for example in one’s own office, helping users to take the simulation more seriously. With this approach, we make it possible to flexibly switch between different environments of different sizes, only requiring developers to adjust the furniture layout. We propose an AR earthquake simulation using a video see-through VR headset, then use real earthquake data and implement a novel AR screen shake technique, which simulates the forces applied to the user’s head by shaking the entire view. We run a user study (n=25), where participants experienced an earthquake both in a VR scene and two AR scenes with and without the AR screen shake technique. Along with a questionnaire, we collected real-time heart rate and balance information from participants for analysis. Our results suggest that both AR scenes offer a more compelling experience compared to the VR scene, and the AR screen shake improved immediacy and was preferred by most participants. This showed us how virtual safety training can greatly benefit from an AR implementation, motivating us to further explore this approach for the case of earthquakes.
@inproceedings{2021-Schirm-EarthquakesAR, author = {Chotchaicharin, Setthawut and Schirm, Johannes and Isoyama, Naoya and Monteiro, Diego Vilela and Uchiyama, Hideaki and Sakata, Nobuchika and Kiyokawa, Kiyoshi}, title = {Compelling AR Earthquake Simulation with AR Screen Shaking}, booktitle = {ICAT-EGVE 2021 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments}, year = {2021}, editor = {Orlosky, Jason and Reiners, Dirk and Weyers, Benjamin}, publisher = {The Eurographics Association}, isbn = {978-3-03868-142-7}, doi = {10.2312/egve.20211330}, issn = {1727-530X}, } - Standardizing Objective Measures of Presence in Immersive Virtual EnvironmentsDoctoral Consortium2021DOI
Presence is the feeling of actually being located in a virtual environment, and has been subject to intensive research as long as virtual reality exists. Nowadays, it is commonly evaluated using subjective measures, which most of the time take the form of questionnaires. However, there have been doubts about whether measures of this type are able to truly capture the actual concept behind presence (if it exists, although this is highly likely). While working on my master’s thesis, I found that it is quite difficult in practice to purposefully employ more objective measures into specific experimental designs. This is why I decided to dedicate myself to contributing ways to make this easier for as many scenarios as possible. My current work involves advanced pattern recognition and general methods from digital signal processing to analyze objective, continuous sensory data like eye-tracking and electromyography.
@inbook{2021-Schirm-ObjectivePresenceVR, author = {Schirm, Johannes}, title = {Standardizing Objective Measures of Presence in Immersive Virtual Environments}, booktitle = {Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems}, year = {2021}, publisher = {Association for Computing Machinery}, isbn = {9781450380959}, doi = {10.1145/3411763.3443429}, address = {New York, NY, USA}, articleno = {71}, numpages = {4}, }
2019 🔗
- Towards an Objective Measure of Presence: Examining Startle Reflexes in a Commercial Virtual Reality GamePosterIn Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts Oct 2019DOI
A large body of literature is concerned with models of presence-the sensory illusion of being part of a virtual scene-but there is still no general agreement on how to measure it objectively and reliably. For the presented study, we applied contemporary theory to measure presence in virtual reality. Thirty-seven participants explored an existing commercial game in order to complete a collection task. Two startle events were naturally embedded in the game progression to evoke physical reactions and head tracking data was collected in response to these events. Subjective presence was recorded using a post-study questionnaire and real-time assessments. Our novel implementation of behavioral measures lead to insights which could inform future presence research: We propose a measure in which startle reflexes are evoked through specific events in the virtual environment, and head tracking data is compared to the range and speed of baseline interactions.
@inproceedings{2019-Schirm-WiP, author = {Schirm, Johannes and Tullius, Gabriela and Habgood, Jacob}, title = {Towards an Objective Measure of Presence: Examining Startle Reflexes in a Commercial Virtual Reality Game}, booktitle = {Extended Abstracts of the Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts}, year = {2019}, publisher = {Association for Computing Machinery}, location = {Barcelona, Spain}, month = oct, isbn = {9781450368711}, pages = {671–678}, doi = {10.1145/3341215.3356263}, address = {New York, NY, USA}, numpages = {8}, } - [DC] Case-studies of Contemporary Presence Theory: Towards More Objective and Reliable Measures of PresenceDoctoral ConsortiumIn 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) Mar 2019DOI
A large body of literature is concerned with models of presence-the sensory illusion of being part of a virtual scene-but there is still no general agreement on how to measure it in an objective and reliable way. When it comes to virtual reality, presence is often considered as one of the main factors contributing to quality of experience, yet existing methods either rely on subjective assessments of users or on specifics of the virtual environment they are applied in, making it difficult for experimental procedures to be generalized. This paper presents ideas for research into promising measures of presence, based on first experiments with novel behavioral measures inside a rich environment which users can feel present in more naturally.
@inproceedings{2019-Schirm-ObjectivePresenceVR, author = {Schirm, Johannes}, title = {[DC] Case-studies of Contemporary Presence Theory: Towards More Objective and Reliable Measures of Presence}, booktitle = {2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)}, year = {2019}, location = {Osaka, Japan}, month = mar, pages = {1363-1364}, doi = {10.1109/VR.2019.8798203}, issn = {2642-5246}, }
2018 🔗
- HCI Experiments Inside Environmental Narrative Games: Expanding the REVEAL FrameworkThesisSep 2018Website
Using existing games to conduct experiments about human-computer interaction makes them not only more fun and natural, but in the case of presence measurement in virtual reality also likely to be more effective. As a contribution to current research in this area, the environmental narrative game framework "REVEAL" was expanded by an experiment system which allows researchers to implement experiment scenarios without wider knowledge of the framework’s implementation, while still collecting all necessary data.
@mastersthesis{2018-Schirm-RevealExperiments, author = {Schirm, Johannes}, title = {HCI Experiments Inside Environmental Narrative Games: Expanding the REVEAL Framework}, year = {2018}, month = sep, url = {https://www.johannes-schirm.de/research/thesis/master-hci-experiments-reveal-framework/}, address = {Sheffield, England, UK}, school = {Sheffield Hallam University}, }
2017 🔗
- Self-avatars in virtual reality: Automated validation of body shape and animation with sparse markersThesisIn Virtuelle und Erweiterte Realität Nov 2017DOI Website
Especially in virtual reality, it is worth the effort to provide a personalized avatar in order to represent the own body. Although it is possible to manually create such an avatar from a 3D scan of every user, this is only feasible for special scenarios. A 3D scan requires not only expensive hardware, but also takes some time for calculation and optimization of the results depending on the desired quality. The main goal of this thesis is to make first steps towards realistically including high-resolution self-avatars into realtime applications for virtual reality and also to develop tools for further optimization of this approach. In doing so, it is built upon an already existing procedure to generate avatars from body measurements and statistical information about human bodies.
@inproceedings{2017-Schirm-SelfAvatarsVR, author = {Schirm, Johannes and Mohler, Betty and Thaler, Anne}, title = {Self-avatars in virtual reality: Automated validation of body shape and animation with sparse markers}, booktitle = {Virtuelle und Erweiterte Realität}, year = {2017}, series = {Berichte aus der Informatik}, publisher = {Shaker}, month = nov, isbn = {978-3-8440-5606-8}, pages = {201--202}, doi = {10.2370/9783844056068}, address = {Aachen, Germany}, department = {Research Group Mohler}, editors = {R. Dörner and R. Kruse and B. Mohler and R. Weller}, event_name = {14. Workshop der GI-Fachgruppe VR/AR}, event_place = {Tübingen, Germany}, issn = {0945-0807}, }