Google Beam Brings Life-Size AI Agents Straight Into the Meeting Room

Google recently opened its labs to show off an experimental AI agent that appears full-size on the Beam telepresence device. Reporters who stepped in for a close look described the figure as strikingly realistic, smiling and moving in ways that made the conversation feel natural.

Beam started as a way to make remote participants appear to be sitting right there with you, accurate to scale and depth. The Dimension, a collaboration with HP, combines six cameras and a powerful server configuration to create a believable 3D presence. Instead of flat video, you get a person on the other end who appears to have just sat down in the same room with you, owing to spatial audio, which places speech exactly where the speakers are onscreen.

The latest trial expands on that by including an AI agent. For the time being, she goes by the name Sophie, and she is a basic yet effective presence. She wears a dark turtleneck top, speaks clearly, and is fluent in several languages. She can even notice her surroundings through the cameras and read writing, look up the weather or a map, and answer basic questions. She even utilizes motions, albeit they can be a little out of rhythm because the system binds them to the music.

Group meetings are now possible with Beam, since remote colleagues coming in from computers or phones via Google Meet appear in full size on the large screen. The system does its best to keep everyone on the same scale and employs positional sound to indicate who is speaking from where. Google has also been looking into possible partnerships with Zoom to provide users with more options. The goal is to make these hybrid meetings feel more balanced and connected, especially when some participants are present in a room while others are dialing in from afar.

The engineers built the agent on cloud servers rather than on a local workstation. So the Beam unit simply catches the environment and sends information back up to the cloud, while the AI responds in real-time. In the current samples, the agent only appears in 2D on the screen, but this still creates a far greater sensation of shared presence than normal video calls. Specifically because of accurate eye contact, natural head tracking, and awareness of physical space.

Some companies have already had early access to the hardware in their workplaces. For the time being, the life-size AI trial is only taking place at Google Labs, with the Mountain View location serving as the primary testing area. That’s where the developers are putting it through its paces, doing extended durability testing and fine-tuning the accuracy of the head tracking. It’s a bit of a guessing game, but the same agent may end up being utilized for a variety of other things, including information desks, guest greeting systems, and any case where consistent availability is required. It may also be used at hotels or theme parks to answer inquiries and guide visitors around without the need for a human service member to be there at all times.
[Source]