Artificial intelligence in smartphones — why do we need it?

The Truth About The Huawei P20 Pro… [Video]

Video uploaded by Unbox Therapy on August 31, 2018

Artificial intelligence in smartphones — why do we need it?

The technology race cannot be completed, but it is becoming more and more difficult to surprise and attract consumers with numerical characteristics. Therefore, over the past couple of years we have seen a lot of experiments in the design and new software capabilities of smartphones. Since the end of 2017, we have started to hear the phrase “artificial intelligence” often. What lived in scientific laboratories 10 years ago, five years ago became public through Google services, and literally appears in our pockets today.

The presence of “artificial intelligence” has become a fashionable and even mandatory characteristic of any current device. Sometimes you wonder how many familiar functions have suddenly become “smart”. However, vendors here can be both scolded and understood, because AI is a very broad concept, for which complex algorithms are not required to hide.

We know that companies like Google have long used machine learning and neural networks to improve mail services, image processing, voice assistants and translators. On smartphones, the application scenarios are generally similar, but they are added to the functions of security and optimization of operation and autonomy of devices.

Cloud VS Device

Users of the same Gmail or Google assistant may ask-if smartphones already have such “smart” services, what has changed over the past year? First of all, some functions can now be implemented directly on devices without using the “cloud”.

For example, Google Photos used to recognize people in photos, and also allowed you to search for images, but the cataloging of photos took place only after they were uploaded to the company’s server. With the advent of chip sets with built-in NPU-module (Neural Processing Unit) image recognition can be implemented directly on the device, which means — more quickly (no time spent on loading and unloading data) and safely (no one will see your photos, even the algorithm on the server). The use of a special module for these tasks allows you to optimize power consumption and speed, because the implementation of such functionality by the CPU and GPU is possible, but not effective.

Huawei was the first to announce the “smartness” of its smartphones, but it is not the only one working in this field. After the announcement of Kirin 970 and Mate 10 (Pro), Apple introduced the A11 Bionic platform with a built-in Neural Engine, and Snapdragon 845 has a Hexagon 685 DSP focused on solving the same problems.

To say whose Artificial Intelligence is better, even in terms of characteristics, is difficult, because some standard and independent measurement tools have not been invented yet, and on different devices’ “intelligence” solves different problems. Therefore, it’s no surprise that when Huawei measures the performance of its GPU compared to the solution from Qualcomm, it turns out to be faster. Nothing prevents it from being acquainted with the capabilities of a particular smartphone that uses AI-functionality.

Let’s see for functions AI can already perform in smartphones.

AI for cameras

“Artificial Intelligence” has long proved its effectiveness in image recognition tasks and it is quite obvious that it is used in smartphone cameras. For example, scene recognition functionality: the Huawei P20 Pro NPU module can identify in real time what is in the frame and choose the most suitable settings for a particular plot.

What gives? Usually, the automation selects the shooting parameters based on illumination, contrast, and other parameters of the scene. Object recognition allows you to improve the algorithms. For those who understand what manual settings are, it is obvious that when shooting a moving object, the first task is to get a sharp picture, so it is necessary to reduce the shutter speed and raise the ISO slightly, sacrificing quality. For shooting a landscape or portrait you need a different set of settings.

The software recognizes the scene and selects the best shooting parameters, or switches the camera to the desired mode. At the same time, photos can be processed — the SOFTWARE will increase the contrast and color saturation, as if the photo has already been applied to the Instagram filter. From a professional photography point of view, a picture like this may seem wrong. To be honest in mobile photography algorithms have always been important to make the photo look great on screen. Accuracy of color transfer should just be left to DSLR cameras.

Current models already recognize dozens of scenes with a set of additional settings depending on the specific shooting conditions. A set of scenes previously “included” in the SOFTWARE, the manufacturer is engaged in the initial training of the neural network (if it is used network), the selection of photos for such training, and so on. This functionality can be further enhanced with classic SOFTWARE updates.

The use of Artificial Intelligence does not end with just one scene recognition. AI can help with “intellectual” auto focus and video stabilization. The work of auto focus is noticeable in the form of small squares, which indicate the focus points on moving objects. For example, they appear on plants swaying in the wind.

AI is used in advanced video stabilization and works in tandem with an optical stabilizer. When shooting Full HD with hands, you can get a smooth picture when walking, comparable to that given by individual stabilizers.

Another example of Artificial Intelligence work is slow motion video (HD @960fps). The function first appeared on Sony smartphones. To achieve the desired result is difficult. You need to catch the moment when you need to click on the descent, and the camera slows down just for a second in real time. This is just enough time for the user to turn on the desired mode, point the camera, and place a special square in the frame area where movement is expected. After clicking on the shutter, the camera determines when the movement will start in the selected area, and slows down the video, to make achieving the result so much easier.

Object recognition allows you to add photo cataloging to your local gallery. Photos are sorted according to the place of shooting, the faces of recognized people, as well as categories (food, documents, and landscapes), and local search.

… for translation

Another scenario of using AI is speech recognition and translation. Power NPU in this case involves an external translator. The application can work as a regular translator, translate text in photos, individual phrases, and even work as a “live” translator for a group of several people.

… for safety

Quite understandable, Artificial intelligence is generally recalled when speaking about safety.

As already mentioned, all of the data on the device does not go beyond the device. The screen unlocks when AI recognizes the user’s face. The digital image (model) of the user’s face is stored in a special encrypted memory area, as well as fingerprints.

… for productivity

In reference to everything related to images, we often talk about already “trained” algorithms — without updates, the camera will not be better to recognize your cat or stop confusing it with a dog (which also happens). At the same time, manufacturers claim that AI allows you to optimize the operation of the device for a specific user.

Remembering the typical sequence of running applications and depending on the time-place, the software is able to accelerate the launch of individual programs in particular and optimize the operation of the smartphone as a whole, which should result in a consistently good speed and a positive impact on autonomy.

Companies claim that AI is also used in noise reduction algorithms in telephone conversations as well as in many other aspects.

What’s Next?

The presence of embedded “smart” functions improves the user experience, but we are talking only about the first wave of AI implementation in smartphones. The next step is to use the potential of the platform by third-party applications. A good tone is the availability of API for developers and support for popular libraries like TensorFlow from Google and Cafe from Facebook. All of this is already there, all that is left is to wait for applications to be created that will show advantages of modern devices. It can be both AR-games and applications for solving individual problems. For now all that’s clear that AI is turning into another important feature of modern smartphones.

Links

• Artificial intelligence in smartphones: Siri vs. Cortana, which is the funniest virtual assistant? – iGotOffer
• AI-powered smartphones and the features that will make you want to buy them – Business Insider
• The Hundred-Page Machine Learning Book by Andriy Burkov – Amazon
• Trade in your cell phone for best cash online – iGotOffer