I was invited by TribecaHacks to CERN in Meyrin, Switzerland, to create new forms of interactive storytelling over the course of five days. The hackathon was called Story Matter. Teams were composed of scientists, storytellers, designers, and technologists. My team decided to create a non-linear sensory-based narrative about climate change told from the perspective of three characters: earth, ocean, and atmosphere.
The experience is controlled by the viewer’s brainwaves. Upon entering the installation, the viewer puts on a MindWave EEG sensor. The viewer sits down and is encapsulated in our tunnel. We constructed the tunnel to block out potential distractors so that the viewer can focus their mind on what they are watching. The video is projected to the back of the tunnel, which has been painted with phosphorescent paint. This gives the image a decaying effect over time.
The video is composed of scenes. Each scenes has 3 thoughts. A thought is composed of a line of text and an image, which is also categorized to a specific topic. While the scene plays, the brainwave activity is constantly monitored. When attention is high and meditation is low, a value of positive 1 is recorded. When attention is low and meditation is high, a value of negative one is recorded. When the scene finishes playing, each thought’s aggregate value is totaled. The next scene is dynamically created with two thoughts from the previously highest ranked category. A random thought is additionally added to the new scene.
After the hackathon, I took the opportunity to dive deeper into the code and build a platform so that the foundation could be reused. In the latest version, videos are used instead of still images. A project can have an unlimited number of topics. Each topic contains individual thoughts, which are short video files. By modifying a text file, a filmmaker can create their own sensory-based narrative. In addition, the sensor monitor and video playback have been separated into two programs.