The study aims to understand the influence on human creativity in the activity of open-ended play with a rule-based AI system in a VR environment. It also aims to understand how the players’ mental model of the AI companion is shaped based on their play experience. The study is run with participants who play both alone and with an AI companion while building castles using 3-d blocks in the VR environment. The participants answer post-play survey for both sessions. Thematic analysis reveals that participants did feel that the AI enhanced their creativity due to three reasons; discovery of new possibilities, AI's skills in building a castle-like aesthetic and players giving up control to the AI when they didn't know how to proceed. The quantitative results show that participants felt positively creative and joyful in both the-alone and with-AI sessions with no significant difference. The player's mental model of the AI companion was influenced based on user expectations and the degree of control they experienced during the gameplay.
Project
Location
Duration
Supervisor
Focus
University of Nottingham
July - September 2021
Dr. Colin Johnson
Human-computer co-creativity, Virtual Reality, Experiment design
Platform
Unity, Oculus Quest 2
Music
Art
Previous Literature
Human - AI
Co-Creativity
Research Gap
Play
Most fundamental activity in human culture
Fosters Creativity
Strengthens social bonds
...enhance the creativity of users who are interested in engaging in explorative processes, with the help of a collaborative AI companion
-
What factors influence human player's perception of an AI that co-creates with them in an open-ended play acivity?
Mental Models
Open ended play
Select object
Drop object
Rotate Object
Grab object
Rotate Object
Rotate base
Rotate base
Delete Object
A thumbs-up gesture was decided upon as a favourable gesture to signal the AI for its turn. This hand-gesture has been metaphorical popular in English-speaking countries as an approval towards something. Given the context of the study and the recruitment of English-speaking participants, the gesture would have a positive connotation, though in different cultures its meaning can be perceived differently (Strazny, 2004)
Strazny, P. (2004, November 1). Encyclopedia Of Linguistics. Fitzroy Dearborn Pub.
When the player wants to signal the computer to take a turn, they lift their left hand give a thumbs-up
The AI recognizes the signal and moves blocks onto a selected location
Once the computer has made its move, it automatically waits for the human’s turn. It is important to note that the human can manipulate the AI’s moves, including deleting its placed blocks.
Rules of play highlighting the set of blocks the AI would place on a lower block if its drop-zone is free
A sample structure created using the rules by which the AI plays. The human player is not limited to these rules and can place any blocks over the drop-zones as long as they are empty and available.
The framework of Gero et al., (2020). has been used to define the global behaviour, knowledge distribution and local behaviour of the AI agent in the current system.
Global behaviour
-
The AI plays with a certain coherence and doesn’t offer any destructive behaviour in the experience of open-ended play
Knowledge distribution
-
The AI plays using certain skills of building a castle like structure.
-
It has no knowledge of other structures or forms.
Local Behaviour
-
The AI plays through player-initiated turn taking offering contribution that is meaningful but not over-bearing
Gero, Katy Ilonka, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, and Maria Ruiz et al. 2020. "Mental Models Of AI Agents In A Cooperative Game Setting". Proceedings Of The 2020 CHI Conference On Human Factors In Computing Systems. doi:10.1145/3313831.3376316.
The procedure was the allow 15 participants to play with the castle building toy in VR across two sessions; build a castle while playing alone (session A) and build a castle while playing with the computer (session B). After each session the participants were given questionnaires to fill.
Session A: Play Alone
( 8 participants)
Survey A
Session A:
Play with AI
Survey B
( 7 participants)
Session B: Play with AI
Survey B
Session A:
Play Alone
Survey A
Survey A
Survey B
Data analysis using a Wilcoxon signed rank test for within subjects. The results suggest no significant difference (all p-values are greater than 0.05).
We can conclude that even a rule-based AI system was effective in making players feel creative in an open-ended play experience. We cannot claim that this was significantly more than that of their experience playing alone. The AI influenced creativity in the following manner:
-
Players discovered new possibilities through the AI’s inputs (knowledge distribution)
-
Players may not have thought of a castle aesthetic as it was inbuilt in the AI (knowledge distribution)
-
Players transferred control to the AI when they had no idea how to proceed (knowledge distribution, local behavior)
Fourteen out of the fifteen participants did not get close to guessing the rules by which the AI was playing the game. The player’s mental model of the AI was shaped based on their expectations from the AI and the level of control they wanted to give or take from the AI.
Play Alone
Play with AI
Play Alone
Play with AI