Full Paper (May 2017)
Abstract:
Can soft skills like critical thinking, creativity, communication, and collaboration be effectively tested in VR? And can VR be used to train those skills in a fun and engaging way? VRIQ is a 5-minute test published to Steam on May 3, 2017 that aims to start answering those questions.
Introduction:
Soft skills like critical thinking, creativity, communication, and collaboration are notoriously difficult to test for, and there are relatively few pieces of software that specifically aim to train those skills. VRIQ starts by focusing on creative problem solving (a purposefully ambiguous term which is meant to describe general problem solving ability). Within the 4 Cs framework, this fits most closely into the “critical thinking” box.
Enrichment of these skills is widely considered by educators to be the most important part of education in the 21st century, since with these skills specific specializations of knowledge can be learned and applied relatively quickly (an important ability in our quickly changing world).
Our specific research question is this: Can a VR test create a normalized distribution of scores, from which an IQ score can be assigned?
Hypothesis:
The scores from VRIQ will form a normalized distribution from which the top 10% and bottom 10% of scores can be identified.
Methods:
VRIQ was posted on Steam May 3, 2017 at 11am:
A link to the Steam page was posted to reddit.com/r/oculus around 11:30am that same day: https://www.reddit.com/r/oculus/comments/6923hk/vriq_making_people_smarter_through_vr/
Unity Analytics events in the app’s code allow us to gather users’ score data. We also instituted some anti-cheat methods to try to make the data more meaningful. All data collection was completely anonymous.
The core app was built in a single day with a team of 3 Unity VR developers, with a few extra days of work put into setting up analytics and Steam hosting.
Score data was collected over the course of 1 week. First-time scores were separated from subsequent scores, as well as sorted by headset.
Results:
The score distribution was fairly symmetrical, but was not a normal distribution. Instead, it had 2 local peaks, 1 on each side of the mean (18 for Vive and Oculus) and median (18 for Vive, 20 for Oculus). Despite this, the spread and fairly symmetrical distribution allowed identification of certain percentage groups.
Discussion:
We are aware that this study will by itself provide limited utility for actual intelligence testing, especially since our data is anonymized and so can’t be compared to peoples regular IQ scores. Our hope is that this gets people thinking about how to test and train soft skills with VR. A follow up study conducted offline with in-person testing could be useful for conducting comparisons of a tester’s VRIQ score to regular IQ score.
If you have thoughts about this test or are interested in collaborating on future projects, comment below or email contact3DIQ@gmail.com
Acknowledgements:
Thank you to Michael Carney, Ken Richlin, and Sky Nite for your work on the VRIQ app. Thank you Solaris Nite for data processing.