Hey Facebook

Voice Platform from Zero to Launch

How do you build a voice assistant platform to compete with other big tech giants?

"Hey Facebook" is the voice command experience similar to "Hey Google" "Alexa" and "Hey Siri". By saying "Hey Facebook", users can control Oculus Quest 2 and Facebook Portal hand-free -- including taking screenshots, opening apps, grouping up with friends, controlling music, playing games, and more.

Behind all these amazing experiences, developers need tools and a robust platform to work on -- Action Console, Alexa Console, and SiriKit. Naturally, Facebook would need a world-class developers' platform to compete with Google, Amazon, and Apple. And that's exactly what I designed from the ground up.

Role & Responsibilities

Lead as the sole designer of Facebook voice developer platform from concept to launch

Why is it Important

Building voice experience is hard. It involves many difficult and novel concepts of machine learning, voice interaction, entity resolutions, agent development, testings, model integration, and more. There are also multiple specialists involved - from developers, linguists, conversation designers to data scientists. To make delightful "Hey Facebook" experiences into reality, a well-integrated, robust, extensible, well-designed developer platform is needed. But when I started, no such thing exists - just a few homemade tools made by developers. Time to get to work.

Naturally, Facebook would need a world-class developers' platform to compete with Google, Amazon, and Apple. And that's exactly what I designed from the ground up.

Outcomes

The first version of the platform was launched in September 2019. It was a very successful MVP launch and people began adopting it in their process. Now, this platform has become THE core tool that everyone at Facebook including external partners uses to build voice experience on Oculus and Portal. This includes calling, getting weather, controlling video, playing games, getting recipes through the SideChef app, and controlling AR effects.

Here are some stats comparing the first half of the year after launch to the next half (because compare it to before launch would be too easy):

85%

grew in weekly active users

20%

reduction in average time to production

77%

tooling satisfaction

55%

of all experience was built in Q1-Q3 alone

30%

increased in development velocity

Thanks Prakarn for being such an amazing designer of Studio. I could not say more thanks for your contribution, from the old days that you are the first designer, you saved our engineers' attempt of trying to be a designer to nowadays, you always listen humbly, act fast and deliver good stuff.
— Facebook Engineer

The Design & Process

Note: Since a majority part of the platform is under NDA, I can only talk about the outline of the design and my process. But I believe the outcome speaks for itself how successful the platform is.

In the early stages, there was little to no design vision and direction. And Facebook wasn't sure that the unified platform was something they wanted to put the big bet on. So the team started with just ~10 engineers and a project manager. After usual discovery, research, prototype, and a lot of collaborations, I designed the platform that adheres to these principles:

  • Design for utility and empowerment
  • Be the single source of truth
  • Allow collaboration between different users’ roles
  • Aim for customizable automation
  • Enable quality through testings and quick iterations
  • Make it easy to integrate to other platforms

Usability is also the key factor. Many of the engineers and other users were new to voice development. To accelerate the onboarding process, the tool itself teaches users how the voice structure breaks down and what artifacts come out in the process. Using a design similar to Wit.ai (another tool I designed for Facebook), the platform was extremely easy for engineers to get on board with just a Bootcamp lab that they can complete within a day.

Thanks a lot for creating such an amazing product and an ecosystem for bootstrapping new experiences and tuning the existing ones. It is literally a 10x improvement comparing to what we had to do 2 years ago to create a demo.
— Facebook Engineer


Hey Facebook

Voice Developer Platform

Facebook + Wit.ai

Product Redesign

Facebook Portal

Interaction Design, Voice, AR

Password required

Novo Nordisk

Software as a Medical Device UX/UI

Skype for Business

Product, Visual, Interaction Design

Microsoft Teams

Design Audit, Mobile App Design

Let's make great things happen together!

Contact Prak