Facebook Portal is a device that aim to connect people together with features such as video calling, Whatsapp, and Messenger. With its AR capability, it allows people to use fun AR effects such as AR masks and background.
As a smart device, using voice to enable AR effect is no brainer. But getting the experience right is the key.
Role & Responsibilities
Senior Product Designer
Voice and UI design
With the old AR photo booth effect, users have to tap the screen to enable the effect. That breaks the flow of the users because they have to be closer to the device to touch the screen or hold onto the remote in the case of Portal TV. And that makes the experience unnatural for group conversation.
Integrating the voice interaction for the AR effect is natural. The device already supports voice commands such as getting weather, playing music, and more. By adding voice interaction to the AR photo booth experience, people can sit on a couch together and ask Portal to put on AR effects.
Additionally, with voice, it allows users to customize the effects which are not intuitive using touch such as making the effect bigger/smaller, switch to a different AR mask, or make the nose bigger.
Just like any voice interaction, the discoverability of the feature is difficult. How would Portal let users know that these effects support voice while others don't? How to convey the voice command so users can customize the effect correctly?
Additionally, after exhaustive research, voice + AR interaction pattern hasn't been established - not even from Apple or Google. This is something new the team has to invent.
John Velasco / Digital Trends
First, for discoverability, without relying on the Portal's user education feature, a "Voice Control" group was added to the list of the effect category. Within the group, the effect itself has a clear voice-enabled badge around it to distinguish them from normal effects. From the research, this was quite an effective way to get user attention to try these effects out.
After users select the effect, we promptly show them the education module. After users give this command, Portal will show the options for users to interact with the effect such as "make my {mouth, eyes, nose} {bigger, smaller} in the face morph or "add a {smile, sad, crying} emoji" in emoji background effect.
Beyond controlling, users can send themed cards and take photos and videos with AR effects to send to friends and family.
After multiple iterations and engineer work, the Voice Control AR Effect was launched on October 6 as reported by Techcrunch along with Netflix and Zoom announcement. The usage has been overwhelmingly positive. The AR voice effect beats multiple core categories in Portal and sends a clear signal of users' demand for more voice interaction like this.
