Interactive Volumetric Live Performance Between Sydney and Tokyo

January 23, 2024

Keio University Graduate School of Media Design (KMD)

For the first time ever, a volumetric video-based live musical performance viewable from any angle was performed to an audience located 7,800km away across the ocean, co-creating improvisational music between the performer in Tokyo and the audience in Sydney. A stable and high-speed communication network, which is required to transfer a high volume of information that cannot be achieved with ordinary lines, was supported by a collaboration of research and education networks in the Asia Pacific.

Keio University Graduate School of Media Design (KMD), Canon Inc., Bascule Inc., and  Mori Building Co., Ltd. have collaborated to deliver a real-time performance using the Research and Education Internet link between TOKYO NODE and International Convention Centre Sydney for a closing session of SIGGRAPH Asia 2023 “Super-Speed Connectivity for Digital Twins” on December 15, 2023.

The audience in Sydney enjoy the performance from Tokyo with point-cloud volumetric video.

Walking around the performer to move the viewpoint.

The audience interacts with the performer in Tokyo.

The violin is performed from the Volumetric Video Studio in TOKYO NODE.

Violin is performed from Tokyo and improvised responding to the audience’s action in Sydney, to co-create the music as participatory music. The Audience in Sydney is able to freely move the viewpoint such as front, side, back, or moving close to the violin.

The Performance was streamed in real-time from the Volumetric Video Studio in TOKYO NODE with 57 cameras in the immersive green screen environment. The captured video was converted to a point-cloud using the proprietary cutting-edge volumetric video technology. 

The SuperSpeed Research and Education connectivity between TOKYO NODE and International Convention Centre Sydney was provided by AARNet, Australia’s Academic and Research Network, ARENA-PAC, Arterial Research and Educational Network in the Asia Pacific operated by WIDE Project, and GOREX, The Guam Open Research and Education eXchange.

Volumetric Video Technology Team

Atsushi Date , Shuntaro Aratani (Canon.Inc)  

Masayoshi Boku, Sakiko Osawa (Bascule) 

Ou Sugiyama , Kazuki Motani (Mori Building) 

Shinji Aoki (NiTRo)

Super Speed Network Team

Steve Maddocks (Director International, AARNet)

Luc Betbeder-Matibet (Director, Research Technology Services, UNSW)

Chris Zane (University of Hawaii / GOREX)

Hirochika Asai, Takashi Tomine, Takumi Ishihara, Mariko Kobayashi (ARENA-PAC / WIDE Project)

Daiya Kato, Shun Arima (KMD)

Keiko Okawa (KMD / WIDE Project)

Performance Team

Yuehui Yang (KMD)

Daichi Ito (IBM)

Yamen Saraiji (SONY AI / ACM SIGGRAPH Hybrid Society adhoc Committee)

Masa Inakage (KMD / ACM SIGGRAPH Hybrid Society adhoc Committee)