Voice Assistant in Mobile Apps: Flipkart

An In-depth analysis of the performance and end user experience of Flipkart's multilingual Voice Assistant available currently inside their flagship Android app.

This blog is a part of a series by Slang Labs, called “Voice Assistants in Mobile Apps”. Here we break down the voice assistant functionalities that have been added by businesses to their mobile applications and analyse them in-depth.

In this edition, we break down the Flipkart mobile app which has integrated an in-app voice assistant to enable multilingual voice search. We believe that it's essential to recognise the trendsetters and understand how they are adding voice assistants inside their applications to cater to the next billion users. This analysis will also help businesses understand the usage of voice assistants in applications.

We have already broken down features of voice assistants in ‘JioMart’, ‘Amazon’, ‘My Jio’, ‘Gaana’, ‘YouTube’, ‘Trainman’ and ‘Paytm Travel’.

Voice is bridging the gap and breaking the digital barriers!

Flipkart understands the importance of these Next Billion Users and is set to reach them first. They know the app UIs are complex and have a limit to which they can be simplified. But to onboard these users, they need to make the User Experience far more simple and comfortable. And here comes the voice. We Indians love talking and it comes naturally to us. We don’t need special training or a crash course to talk.

First Time launch of Flipkart Voice Assistant
First Time launch of Flipkart Voice Assistant

Flipkart and other e-commerce brands in India understand the impact of voice. This is why these brands either have an in-app voice assistant or are in the progress of integrating one.

Visual Breakdown

Flipkart Voice Assistant Analysis by Slang Labs
Flipkart's Voice Assistant

Mic icon in Standby Mode

The mic icon is present in the traditional way. It is in the top right corner of the screen lying beside the search bar. This increases the visibility of the icon and also indicates that the user can alternatively press the mic button to search.

Flipkart’s Voice Assistant Surface

Flipkart, just like Amazon has kept its surface almost button-less with just a cross icon to close the surface. Upon tapping the mic icon, the surface pops up at the bottom of the screen with a curved shape on the top. The surface upon start contains a text message that says “Listening” and a wave-like pattern below it indicating that it is well… listening.

An alternative approach to this is what we saw in the in-app Voice Assistants by Udaan and JioMart. They allowed the user to change language, mute the assistant and tweak other settings.

Visual Experience

The voice assistant surface instantly fires up upon clicking on the mic button. It is in the listening stage and replaces the default “Listening” text with the user utterance. One time I said “I know you are listening buddy” and it showed me body washes. Ain’t it fun Eh?

The surface has a very clear UI and does its work to let the user know that the assistant is listening. The visual hints to guide the user are missing in this version. These hints could have helped the user understand the possible utterances.

Functional Breakdown

Multlingual Voice Search in Flipkart Voice Assistant
Multilingual Voice Search in Flipkart Voice Assistant

Voice Search

Flipkart’s Voice Assistant accurately understands the utterances in most cases. You don’t have to speak awkwardly slowly for it to understand. The assistant today is capable of understanding the utterances in English and Hindi. YES! It is multilingual!

Voice Action 

The assistant as of today won’t do much when you ask it “Add 20 oreo biscuits to cart” except showing the page with search results for Oreo Biscuits (and honestly the strawberry ones taste much better than we expect them to be).

Voice Navigation

The voice assistant doesn’t support navigation. Utterances like “Take me to cart” or “Go Home” results in an error notifying the user that the assistant couldn’t understand the utterance.

C̶o̶n̶t̶i̶n̶u̶e̶d̶ Instructional Conversation

This assistant upon accurately understanding the utterance takes the user to the search results page and disappears. So, if a user needs to voice search again, they will have to tap on the mic again. The assistant doesn’t acknowledge after accurately understanding the user's utterances so it feels like a very one-sided conversation (just like me at 3 AM 👀).

Limitations of Flipkart’s Voice Assistant

Language Barrier

The assistant today supports English and Hindi. Support for regional languages would have made the experience much better for people who aren't fluent with English or Hindi. The assistant also doesn’t allow the user to change the language of the assistant without changing the “App Language” from the side menu.

Absence of Onboarding Flow

The voice assistant is built with an aim to onboard those users who are unfamiliar with the app interface and English-only experience. The absence of an onboarding flow to guide the users makes the experience unpredictable for these users.

To read more about the onboarding flows and how they can be designed to guide these new users, read our analysis here.

Error Handling

The assistant upon failing to understand the user utterance simply ends with a beep sound. The assistant today doesn’t speak out that it has trouble understanding the utterance. Although it does display an utterance it understands like “Show me black t-shirt for men” thus explaining to the user what they should try next time.

Conversation Design

As discussed above, the assistant never speaks back. The experience feels like one-way communication which often feels a bit odd. My grandfather accurately described the experience as “ये वापस कुछ बोलता क्यूं नहीं?” (Why doesn't this say anything back?).

EndNote

Flipkart’s voice assistant reduces the searching time significantly. It is able to understand most of the utterances accurately and displays the search results as expected. Although there are a few things missing from the User Experience point of view, the assistant seems to do a good job.

We released an e-book titled “Voice Commerce in India” which discusses the approach of popular e-commerce brands towards Voice Technology and what the future looks like for others.

If you want to add an in-app voice assistant, we have built voice assistants for e-commerce and travel domains. They can be integrated with your android app and website in a matter of hours. If you want a voice assistant for any other domain, let’s talk.