Game Changer: With augmented reality the best seats in the stadium were 600 miles away

It’s the bottom of the 4th quarter and your team, trailing by 1, grabs the rebound for a fast break. The floor thrums beneath you as the point guard strides across the court, the footsteps punctuated by the staccato squeaking of his sneakers. You watch as he passes it off to an awaiting teammate in the air, who slams it down the hoop two-handed. As he lands with a resolute whomp the ground shudders, and everybody around you erupts into cheers.

That’d be an amazing courtside basketball experience—except this happened halfway across the country from the game, in real time.

This reflects the new world of haptics envisioned by teams of academics, creatives and product designers based in Tokyo, Japan. One of its thought leaders is Kouta Minamizawa, professor at Keio University in the Graduate School of Media Design. 

The basketball experience took place in the beginning of 2018 at two locales; the actual game in Kumamoto, deep in western island of Kyushu; and an interactive experience of the game held in Tokyo, more than 600 miles away. Headed by Fujitsu and B.League (the Japanese equivalent of the NBA), Minamizawa and his team from KMD (Keio University Graduate School of Media Design)  were brought in as haptic technical advisors to bring to interactive experience to fruition.

The word “haptic” might evoke humble applications, like the innocuous vibrations of your smart phone, the buzz of your controller when you crash into a wall in Super Mario Kart—or if you’re keeping up on the VR/AR trend, haptic feedback sensors on suits and gloves. But haptics, as Minamizawa sees it, is so much more.

“I used to do research on VR, and the thing about VR is that it’s catered to a single person’s experience as opposed to sharing with others,” said Minamizawa. “We are interested in finding ways for people to share experiences together. Fujitsu was also looking for the same thing—sharing experiences. I want to create the type of community or society where we can overcome distance barriers.”

Kumamoto prefecture, having experienced a devastating earthquake in 2016, was still recovering, and B.League decided to hold an all-star game to support the region by generating nationwide buzz and bringing people closer together. Hence, the idea of a live experience integrating the latest in visual, audio, lighting and haptic tech. 

At the 500-person event hall in Tokyo, a mammoth 550-inch screen televised the event in crisp 4k resolution. What really brought home the visceral sensations, however, were the extensive sound pickup and force feedback sensors, mounted in the actual basketball court. 52 microphones were set under, near and above the basketball court—so that every minute sound, from the light dribbles to the trundling footsteps of players were picked up, then transmitted to 30 speakers in the event hall.

In particular, Minamizawa and team worked with Fujitsu on the haptic feedback delivery, installing 8 haptic sensors directly underneath the court. A specific area was set up so attendees could experience the movement on the court in real-time. As basketball players got “closer,” fans could feel the vibrations of footsteps getting stronger, and fans could feel the frenetic movement on the court floor more intimately than even from the stands at the actual stadium.

With all the sights, sounds and kinetic movement painstakingly recreated, there comes an inevitable bottleneck—data bandwidth, and to that end Fujitsu used its own proprietary tech that enabled for speeds up to 1 Gb/s. “Data transfer speed is a key element for haptics to have interactive/natural/mutual communications,” said Minamizawa, which makes the upcoming 5G technology (touting speeds of 20 Gb/s) promising. “With 5G’s higher speeds, it opens the door for even more enhanced experiences.”

The evolution in haptics not only comes from the advancements in technology from the discipline itself—Minamizawa and team attributes miniaturization to that end—but also data science, aesthetics, human psychology, making haptics a multi-disciplined endeavor. “For a great idea, what’s important is to connect with people from completely different fields to yourself.”

When siloed, haptics could only go so far. It’s why Minamizawa has partnered with Loftwork, which acts as a nexus for incubating, spawning and refining new products as well as interactive experiences—bringing concepts from the lab and the theoretical into reality.

One of its collaborative efforts is Living Lab Shibuya, a research hub nestled in the bustling Shibuya neighborhood, an area in process of reinventing itself as the Silicon Valley of Japan. In addition to R&D, it shares research insights with industry experts, artists and creators through live demonstrations.

“[Haptics] was a very small community solely within the university only 10 years ago, and the people involved were just on a very technical level. Now, it is more accessible for people outside of the small community, thanks in part to the connections through Loftwork.”

For example, MTRL is a Loftwork material innovation platform that brings together designers, researchers, artists and material engineering companies together. When Minamizawa’s researchers are prototyping a new suit, for example, and need a fabric with just the right conductivity or flexibility, it consults with MTRL to source the right contacts for the ideal material—connections an academically-focused institution may lack. Conversely, a manufacturer can approach Loftwork with a UI challenge it’s facing, thus drawing upon its pool of designers, scientists and engineers that are gurus in the field.

For budding haptic designers, they’re highlighted and recognized in the Haptic Design Project, showcasing some of the most promising advancements in the field. While many cutting-edge companies tend to monopolize knowledge, Minamizawa’s approach is surprisingly communal and cooperative.

“I want to make it easier for people to grasp what haptics are, and to bring ideas to industries such as entertainment and medical,” said Minamizawa. “Because I want to apply the knowledge from the haptic academic community into everyday life.”

With the technology in place, it’s already becoming a reality—what’s missing is only how more need to play ball.

See the video of B. League below to get a taste of what it was like being at the augmented reality event :