Merging of VR and haptic design
Project Professor Tetsuya Mizuguchi + Rhizomatiks + KMD “Synesthesia Suit”
As part of the Embodied Media Project, Associate Professor Kouta Minamizawa’s team utilized haptic (tactile sensation) research in their creation of the “Synesthesia Suit” to amplify the experience of the VR (virtual reality) video game “Rez Infinite.” Project Professor Tetsuya Mizuguchi, creator of “Rez Infinite,” set the project in motion by proposing that Associate Professor Minamizawa’s team, which researches haptics (tactile sensation technology) at KMD, and Rhizomatiks, a media arts creative group, jointly develop a suit to enhance synesthesia.
TactileHaptic sense is set to grow in significance, as realistic 3D renderings, which owe their higher visual and auditory resolu tions to advancements in technologies such as VR and AR (augmented reality), become the norm. Inside the “Synesthesia Suit,” 26 actuators vibrate against the body to a rhythm, such as the “boom-tss” of a bass drum and hi-hat cymbals, when synchronizing with music, or each actuator vibrates with time variance so the wearer feels a sensation running through the arms and body. Furthermore, the LEDs mounted on the Suit’s exterior light up in sync with the vibrations, extending the VR worldview not only to the wearer, but to the people around, as well.
When the background music and sound effects of the VR work are relayed as trembling, music is not only heard through the ears, but as the completely new experience of full-body tactilehaptic sensation. The subtle trembling created as a result of repeated trial and error sets this apart from simple vibration. This type of haptic design is thought to be the first experiment of its kind in the world.
The Synesthesia Suit was first unveiled in December 2015 on the PlayStation Experience 2015 stage in San Francisco. All project members traveled to see Project Professor Mizuguchi don the Suit to demonstrate playing Rez Infinite as part of his stage presentation. In Japan, the Suit made its public debut during Media Ambition Tokyo 2016 at Roppongi Hills in February and March 2016, followed by other events, including Tokyo Game Show 2016, SIGGRAPH 2016, and the Sundance Film Festival in January 2017 in the U.S. The Synesthesia Suit continues to evolve as more people experience it.
“With this project, we fully established the new field of haptics design. Synesthesia as experienced with the Synesthesia Suit may become commonplace in the near future. One of KMD’s ideals as media innovator is to break ground in new fields and become the first creator.” (Associate Professor Kouta Minamizawa)
(This article was written in March 2017.)
The LED colored lights embedded on the outside of the suit coordinate with the vibra tions felt by the wearer so the VR experience extends to the people around as well. The colored LEDs that light up according to vibra tion, gradation, fade out pattern variations, and other renditions were explored.
The Rez Infinite Synesthesia Suit was exhibited as media artwork at Media Ambition Tokyo, an event held at Roppongi Hills (Tokyo) in February 2016. Accompanied by visuals, light, color, vibrations, and 3D acoustics, the VR experience of one person extends to the entire surrounding space. Oscillators and LEDs built into the venue’s seats made the audience part of the experience.
Ayahiko Sato of Rhizomatiks (left) and Yukari Konishi of the master’s program at KMD
