"Hey Facebook" is the voice command experience similar to "Hey Google" "Alexa" and "Hey Siri". By saying "Hey Facebook", users can control Oculus Quest 2 and Facebook Portal hand-free -- including taking screenshots, opening apps, grouping up with friends, controlling music, playing games, and more.
Behind all these amazing experiences, developers need tools and a robust platform to work on -- Action Console, Alexa Console, and SiriKit. Naturally, Facebook would need a world-class developers' platform to compete with Google, Amazon, and Apple. And that's exactly what I designed from the ground up.
Role & Responsibilities
Lead as the sole designer of Facebook voice developer platform from concept to launch
Building voice experience is hard. It involves many difficult and novel concepts of machine learning, voice interaction, entity resolutions, agent development, testings, model integration, and more. There are also multiple specialists involved - from developers, linguists, conversation designers to data scientists. To make delightful "Hey Facebook" experiences into reality, a well-integrated, robust, extensible, well-designed developer platform is needed. But when I started, no such thing exists - just a few homemade tools made by developers. Time to get to work.
Naturally, Facebook would need a world-class developers' platform to compete with Google, Amazon, and Apple. And that's exactly what I designed from the ground up.
The first version of the platform was launched in September 2019. It was a very successful MVP launch and people began adopting it in their process. Now, this platform has become THE core tool that everyone at Facebook including external partners uses to build voice experience on Oculus and Portal. This includes calling, getting weather, controlling video, playing games, getting recipes through the SideChef app, and controlling AR effects.
Here are some stats comparing the first half of the year after launch to the next half (because compare it to before launch would be too easy):
grew in weekly active users
reduction in average time to production
tooling satisfaction
of all experience was built in Q1-Q3 alone
increased in development velocity
Assistant Developer platform as part of Wit.ai announcement at F8 2021
Thanks Prakarn for being such an amazing designer of Studio. I could not say more thanks for your contribution, from the old days that you are the first designer, you saved our engineers' attempt of trying to be a designer to nowadays, you always listen humbly, act fast and deliver good stuff.
— Facebook Engineer
Note: Since a majority part of the platform is under NDA, I can only talk about the outline of the design and my process. To check out the customer-facing side of this platform, check out wit.ai
In the early stages, there was little to no design vision and direction. And Facebook wasn't sure that the unified platform was something they wanted to put the big bet on. So the team started with just ~10 engineers and a project manager. After usual discovery, research, prototype, and a lot of collaborations, I designed the platform that adheres to these principles:
Usability is also the key factor. Many of the engineers and other users were new to voice development. To accelerate the onboarding process, the tool itself teaches users how the voice structure breaks down and what artifacts come out in the process. Using a design similar to Wit.ai (another tool I designed for Facebook), the platform was extremely easy for engineers to get on board with just a Bootcamp lab that they can complete within a day.
Thanks a lot for creating such an amazing product and an ecosystem for bootstrapping new experiences and tuning the existing ones. It is literally a 10x improvement comparing to what we had to do 2 years ago to create a demo.
— Facebook Engineer
