Scaling with more shades reduces flaws in the classification of darker skins and makes facial recognition technology more inclusive.

The news that Google, from Alphabet, is taking a step forward by developing a new feature in facial recognition technology is undoubtedly cause for celebration. In partnership with Harvard University, the  technology giant is adopting a ten-tone scale to improve the artificial intelligence (AI) system and correct bias problems in its applications and other products.

The new scale, Monk Skin Tone Scale (MST), developed by Harvard University’s  department of sociology  , replaces the scales of the Fitzpatrick Skin Type, a standard used so far to categorize skin tones. The expansion of the palette will make it possible to refine the research by reducing classification failures, especially of darker skins, making the technology more inclusive.

Skin tone rating scales are used in check-in systems such as access passwords, to improve the shopping experience, as well as in more complex applications such as heart rate sensors in smartwatches, artificial intelligence systems including facial recognition applications developed for  skin cancer detection , vision systems used in self-driving cars, and even image filters.

The use of a reduced scale reflects in the selection of information used to train the algorithms. Considering the data sets we have still show a large  predominance of white people of the male gender, the result brings biases for darker skin types and generates errors in the classification by skin tone and gender.

This is because facial recognition technology works similarly to our brain. When we see someone, we send signals of the person’s facial patterns to our brain and it combines or compares them with the patterns already recorded in our memory to make the identification. The facial biometrics software uses the same process, detecting the pattern of the face and comparing it with those registered in the database. Patterns are used to learn how to analyze and identify those. For this, computers divide the face into the so-called nodal points, such as the distance between the eyes, the width of the nose, and the depth of the eye, among other characteristics. This analysis is refined with the technology of deep learning, a strand of AI, in which each time a comparison is carried out it repeats the learning process by recording more data and establishing new connections, making the identification process even more accurate.

 

Excessive possibilities also cause inconsistency

On the other hand, excessive possibilities can also lead to inconsistent results. Thus, the number of ten skin tones on the MST scale was defined based on research to reach a number that considers the criteria of diversity and ease in managing the application to evaluate and train AI systems.

To get to the optimal amount of skin tones on the scale, the Harvard team used digital art tools and interviewed a universe of 3,000 people in the United States. According to the survey, for most people it was possible to find their skin tone on a scale with ten points, and increasing the options did not change the result.

Creating a new skin tone scale is just a first step; the next challenge is to integrate it with the applications and systems we use. This process also involves cultural and political aspects. The different traits and tones prevalent in each country may require, for example, an adjustment based on each geographic region.

The adoption of a broader and more inclusive scale to classify skin tones is an important advance, but there are still sensitive issues around the use of AI. Addressing the equity of skin tone in technology is undoubtedly a great advance, which more than technical aspects, involves social issues and therefore requires the participation of experts in various segments. The MST Scale will certainly improve inclusion in the technology, with greater equity in the use of various products.