The Uncanny Valley of ‘Eye Gaze Redirect’ AI

The Nvidia ‘eye gaze redirect’ feature has been causing a stir. Apple also has this option for FaceTime.

In essence, this application of AI redirects your eye gaze to make it appear as though you are looking into the lens at all times when you are on video. 

At first glance, this might seem like a total no-brainer since eye gaze is so tricky on video. When we pull the hood back however, we can see some real issues with this significant adjustment to something that is in fact, highly nuanced, and personal. 

The way we express with our eyes is idiosyncratic, meaning that everyone has a different ‘eye contact cadence’. The following are just a smattering of examples of how people vary their eye gaze, with each person having their own flavor.

👁️ Blink rate 
While there are average blink rates, people blink differently and at a different rate, there are also a number of nonverbal cues that can be derived from changes in someone’s blink rate).

👀 Looking away and to the left/right/up or down
Whether it’s to the left, to the right, upwards and to the right etc. people have unique habits around these that add flavor to our facial expressions and delivery.

😳 😏 Eye widening or narrowing (Squinting)
To express interest, surprise, shock, slyness/mischievousness etc., the way we widen, narrow or squint our eyes is a critical component of expressing that clearly. 

Unsurprisingly, current AI eye gaze redirect technology is binary – adjusting eye gaze to look directly into the lens. No matter what. 

This presents a big problem because we are losing all of the above eye gaze cues, and more. The tech doesn’t know if you intended to look away or not, but it will keep your eyes fixed into the lens. 

This is creepy. (Take a look at the couple of images below for examples)

There are really only three reasons* why one would wish to not break eye contact with someone, and they are: 

  1. If you are attracted/amorous of someone (you literally can’t take your eyes off of them)
  2. If you are under threat (You won’t allow your enemy out of sight)
  3. If you are about to attack (You won’t take your eyes off of your target)

*shudder*

None of those sound like appropriate behaviors during a professional video call. 

This is troublesome…and we haven’t even touched the ethical issue of deception. If I am talking to someone who is distracted or multitasking as shown above…I want to know. This is critical information as I am spending time building a relationship over video. 

Will this tech evolve? Necessarily. 

But for now, at Virtual Sapiens, we like to focus on using AI to help the actual humans develop skills they can own fully, helping them be intentional about when they look into the lens, and when they don’t.

Avoid the uncanny valley of unblinking, arbitrary, direct eye gaze and instead, learn how to enhance your personal eye contact cadence for both in-person and video based interactions. 

 

*Source: Desmond Morris, ‘People Watching’