Designing a narrative within a user-controlled world that is compelling enough for participants to easily follow the story arc was a challenge in our project to help players experience the social impacts of online hate speech.
Our team at the Department of Internal Affairs (DIA) conducted user testing early and often to identify the problems we needed to solve for the Online Social Norms VR tool.
About the Online Social Norms VR tool
The Online Social Norms VR tool is an immersive virtual reality (VR) experience that demonstrates how the digital environment amplifies and distorts content and experiences.
It aims to support communication and understanding about the impacts of destructive social norms, using hate speech as an initial focus.
The tool is being used by senior leaders and policy decision makers in NZ government to experience:
- what online hate speech looks like
- the perspectives of both the perpetrator and the receiver of online hate speech
- insights into the lives and motivations of people in these situations
- the significant impact online hate speech can have on people’s lives
- the destructive impact online disinhibition has on digital social norms.
This is the first time we’ve used VR to explore how emerging technology might inform policy development through structuring interactive storytelling within an immersive environment.
Immersive storytelling is different from traditional storytelling in that the participant in VR can adopt the viewpoints of protagonists rather than just observe them.
The participant can also influence what happens by controlling what they look at, where and when they move, and which objects or characters they interact with. The experience can be different for each participant.
The Online Social Norms VR experience
The story follows 2 main characters, the perpetrator, Sean, and the receiver, John, of the hate speech. They interact with each other online but have never met in person.
The player moves between the scenes — Sean’s bedroom and John’s living room — first observing each character from the outside, and then stepping in turn into each character to ‘become’ them.
Once assuming the role of one of the characters, the player is able to send and receive online messages through the avatars’ devices, using different social media channels.
The player moves between the perpetrator and the victim, experiencing being both the sender and the receiver of the hate speech.
By enabling the player to become both characters, they come to understand both sides of the story.
User testing the early prototype
In August 2019, after building and assembling the test environment of the characters and 3D spaces, we began user testing the story in the VR experience.
The early prototype used geometric shapes to indicate the locations of characters and objects in the story.
We tested and iterated throughout the development of the tool, wanting to identify what made sense and what was confusing to the player. Test subjects were asked to ‘think out loud’ for the duration of the test experience.
Our 2 testers took notes of the players’ responses during the test sessions and we used these notes to synthesise the findings.
After each session, we spent a further 10-15 minutes with the test subjects, using a semi-structured interview to discuss their interpretations of the story, the characters and the online hate speech.
The user testing identified problems in the interaction design — these included players not understanding the storyline and characters.
Solving the challenges in the story arc
To make sure players had the intended experience and were able to easily follow the story arc, we developed a strong story line and characters, creating a distinctive ambience for each avatar’s world.
1. Story line
The test participants found the story superficial and confusing. They had difficulty in interpreting the story, which jumped around too much in time, and in understanding the relationship between the 2 main characters.
People questioned whether the 2 avatars were both in the same house, and if they were both ‘the baddies’.
This helped us identify that we needed to do a better job of the story sequencing of the VR experience.
Iteration after user testing
Linear story sequence
We remapped the story to a linear structure which made more sense to the player. This meant adding an extra scene. Testing confirmed players understood the story arc.
Test subjects found the characters shallow and clichéd. No one cared or connected with these characters, and with the story lacking warmth and depth the players felt little to no emotional engagement.
The test subjects also found it challenging moving quickly between 2 polar opposite characters to adopt their different viewpoints. This required the player to suddenly make a complete mental shift in perspective, but with these underdeveloped characters it made it hard for the players to feel the difference.
We understood we needed to find ways to build empathy and engage the players so that they connected more deeply with the characters and were able to switch perspectives more easily.
Iteration after user testing
We established a strong story arc with believable characters using 3 techniques.
We created backstories for each character, giving them names, families and personalities.
We added additional props in the VR environment to help reinforce the characters and the world they lived in. The décor and state of the rooms, and the objects in them, helped to give a clear sense of each character’s backstory without overstating it.
Using visual cues, we set the pace and timing of the social media messages the players send and receive when they became Sean or John — this was important for creating different emotional reactions in the player.
The cues guide player interactions and spark the build-up of online hate messages, causing the messaging to intensify and spread from 4Chan to expand exponentially on Twitter.