Alongside the Galaxy S8 and S8+, Samsung also revealed more about its new digital assistant, Bixby. It is deeply integrated into Samsung’s latest flagship phones, but what exactly is Bixby, and how exactly does it work? Also, why do we need Bixby anyway?
What is Bixby?
According to Samsung, Bixby is a digital assistant that was designed such that anything that you can currently do with touch, you can do it via voice with Bixby. The idea is that you can tap, type, and talk seamlessly when performing any task on a smartphone with Bixby.
There are three major aspects of Bixby: a voice enabled digital assistant, Bixby Home where the situational cards are located, and Bixby Vision which lives in the camera app.
How does Bixby work?
To enable the voice prompt for Bixby, users need to long press the dedicated Bixby button on the left side; a short press of the button simply brings users to the Bixby Home screen. Bixby is contextually aware, and is able to work with all Samsung apps as well as a handful of third-party apps (YouTube, Twitter, and Uber are among them at launch). More apps will be supported as and when Samsung releases the SDK for developers, so that may take some time.
Bixby is designed with a high cognitive tolerance and natural language processing. This not only allows Bixby to understand conversational sentences (similar to Google Assistant and Siri), but also be able to act on commands like “show me the last photo I took”, or “beam this video to my TV” (while playing a video on the phone, and your connected TV is in the same WiFi network).
Over time, Bixby will learn from user behaviour, popping up notification “cards” not too dissimilar to Google Now cards. These cards will also appear on Bixby Home, which takes the place of the Briefing news aggregation app located on the leftmost home screen. These cards appear only when useful, which will then be replaced by other contextually useful data. When there are little cards to show, Bixby Home will plug into the Briefing app to display the latest news.
Then there’s Bixby Vision, which is integrated into the camera app. When turned on, it detects what’s in the frame and offers additional info if you need them. You can search for similar images online, or shop for that particular product directly via Bixby Vision.
Besides that, if you’re using Bixby Vision on a bottle of wine, there’s a special “Wine” button that when tapped, shows what year of that particular wine has a higher rating online, and what food is it best paired with.
Does Bixby work?
From our brief hands on with the device as well as in the product demos, the answer is a yes and no. We couldn’t try out the voice-enabled smart assistant portion, but from the demos at the pre-event briefing, that works flawlessly. On the other hand, Bixby Vision was a little more finicky. It incorrectly detected a bottle of Coca-Cola as wine at one point, though the related image search feature did work really well.
In theory, Bixby will only get more powerful over time as it learns the user’s behaviour, but without a wider third-party app support Bixby isn’t as useful as Google Assistant in helping with queries or tasks that are outside the phone (search queries etc).
For a first time effort, Bixby is definitely polished with a solid UI. It just needs the arms into other apps, which will take quite a bit of time depending how easy it is for developers to add support for Bixby in their apps.
So why is Bixby even here when there is Google Assistant?
That is of course, the most important question. It is more of a strategic move from Samsung than anything else. Google’s offering is miles better, and integrates into Google’s entire suite of apps, so why even try with Bixby?
For years now, smartphone manufacturers using Android as their OS have dreaded the day when Google decides to make its own hardware. The Nexus line was one such effort, but it still was a co-developed product with a hardware partner such as HTC or LG. Google marketed them as “pure” Android devices, and only made them in small quantities, making them more desirable.
Essentially, these companies would be in trouble if one day Google decides to do any one of the following:
- Google makes its own smartphones.
- Google imposes a licensing charge to use Google apps on Android.
- Google makes some Android features exclusive to its smartphones.
- Google doesn’t allow manufacturers to contribute to further fragmentation by not allowing Android skins to run on top of Android.
As it stands, two out of the four scenarios have already taken place when Google announced its Pixel smartphones last year. Despite being manufactured by HTC, the Pixel and Pixel XL phones bore the Google logo – a huge tell. And, they were the only two phones to have Google Assistant on Android 7.0 for several months, before being rolled out to other Nougat devices.
These are all telling signs of Google’s increasing ambitions to develop its own hardware. As it is the company already has a hardware division run by the former head of Motorola.
If Google does decide to pull its power and deny features to its hardware partners dependent on Android, the effects are catastrophic – especially for a company with the largest global market share of Android smartphones.
That’s why Samsung had been actively working on producing its own apps even if Android already has equivalent, built-in Google Apps with the same functionality. That explains the development of S Voice, as well as apps such as S Health and S Planner. If one day Google says no, Samsung already has the apps to shore up.
The same goes for Bixby. Samsung designed it to be more of a “smart assistant” that makes navigating and performing tasks on a smartphone more convenient by voice. Core settings can easily be toggled using Bixby, but that’s really about it for now.
In its current state, Bixby certainly isn’t very useful, but it is a powerful gambit should events in the future go against Samsung and all other smartphone manufacturers whose fortunes are dependent on Android and Google.
The challenge now for Samsung is to encourage users to actually use it.
Nicely written article. Kudos.
A suite of in-house built apps should events in the future go against, but was created specifically and optimized for Android. So what happens if Android or “Google doesn’t allow manufacturers to contribute to further fragmentation by not allowing Android skins to run on top of Android.” or in extreme situation not allow any customization or apps by manufacturers? Still not a solution IMHO. It’s nice if you said it’s an alternative, but the gambit reasoning doesn’t adds up too much in this case. In a way, Bixby is a skin by extension to GraceUX or whatcamacallit. If Google said no, that’s it.