Eye Tracking that Reads Emotions!

 

Feelings drive much of our behaviour and so emotional engagement with digital content can be a strong indicator of its effectiveness at getting our attention. This is especially critical for mobile gaming apps as unless users are engaged they may lose interest and delete the app from their phone. Imagine then the power of emotion-enabled gaming apps that recognise emotional responses in real-time and where these same emotions can modify games dynamically to create a more engaging and exciting user experience.

This is now becoming a reality with eye tracking solutions that use webcams and device cameras to monitor emotional engagement. These solutions use vision algorithms to identify key landmarks on the face, such as the tip of the nose or the corners of the eyes and the mouth. Machine learning algorithms then analyse pixels in those regions to classify facial expressions based upon pixel colour, texture and gradient.

Combinations of facial expressions are then mapped to emotions and data is processed in real-time through SDKs. This data can usually be accessed via APIs or visualised in a dashboard. Indeed, eye tracking solution Affectiva’s SDK integrates into user interfaces to allow developers to emotion-enable their apps and digital experiences so they adapt to people’s emotions in real-time.

This creates new opportunities to optimize digital content in a more sensitive and engaging way.  Apart from game design, the technology is also especially valuable for usability and market testing,  video and ad evaluation,  and website design optimization in general.

Interestingly though research by Nielsen indicates that facial coding used in isolation is a poor predictor of sales. EEG research that tracks brain waves is a more powerful indicator of sales. This is why it’s important to use emotional engagement measures along side eye gaze and other research methods to give you a better understanding of the impact of visual content.

One advantage of eye tracking though is that the eye doesn’t lie. We can’t control what we initially look at when we see an image. Indeed, research by eye-tracking company  Sticky.ad  has discovered a gap between what people say they were looking at when they first glance at an image and where their gaze is actually focused.  Self reporting is an unreliable method of identifying what people look at partly because our conscious brain filters out a lot of the information that our visual cortex processes. Psychologists have also discovered we also don’t have full access to what motivates a lot of our behaviour.

At the same time usability testing and Voice of Customer solutions can add to our understanding of how people interact with a user interface and provide insights into the issues they encounter. Each method of research provides a different perspective on how people respond to visual content and tend are most powerful when combined with each other.

Is There A Correlation Between Eye Gaze and Mouse Movements?

 

Some visual analytical solutions claim their is a high correlation between mouse movement and eye gaze, but the evidence does not support this. The highest published correlation is  69% (Lynne Cooke), but research by Google ( see page 29) indicates it is more in the region of 42% and according to an anecdotal study it is only 32%. However,  Simple usability claim it may be even lower, if not non-existent,  when visitors begin to scroll. This is because people tend to scroll and scan content with their mouse either being static (because they use the scroll-wheel) or their mouse follows the right-hand scroll bar. You can almost guarantee that users won’t be looking at where their mouse is when they scroll down a page.

 

Source: sticky.ad/

The “F” Shape:

 

 

Many eye tracking studies have shown that users tend to scan webpages in an “F” shaped pattern. This is partly because in the West at least we read from left to right. As a result we mainly focus on the top left side of the screen and scan across horizontally. Our eyes then scan down the page, again mainly down the left to find what we are looking for. Of course this process can be disrupted by content that grabs our attention or design elements that separate content. Web pages for instance that appear to end at the bottom of the screen won’t encourage further exploration and so engagement often falls dramatically around the fold area.

Google Golden Triangle & Beyond:

In 2005 an eye tracking study by Google and marketing services companies Enquiro and Didit.com, discovered that users tend to view Google search results in more of a triangular shape. Thus a majority of a user’s eye gaze is focused in a triangle at the top of the search results page. This became known as the “golden triangle”.

 

However, Google conducted a further eye tracking study in 2015 which was published by Mediative.  This indicated that more visitors are moving outside of the ‘Golden Triangle’ and that the rise of the mobile user is also changing browsing behaviour.

 

The latest Google study shows:

• Mobile users are habitually conditioned to scan vertically more than horizontally.
• Users are viewing more search results within a single session but are spending less time browsing each one (on average 1.17 seconds compared to 2 seconds in 2005).
• Results that are positioned lower down the SERP (particularly positions 2 to 4) are receiving more clicks than they did a few years ago.

What this tells us it that we can’t rely on generic eye tracking studies to inform us where visitor gaze is concentrated, we need to undertake our own studies where we can. Otherwise we will making decisions based upon assumptions that may well not be valid.

Observer Effect:

The Hawthorne effect refers to the fact that people often change their behaviour when they are aware they are being observed. This is why TV viewing research panels ignore what new participants watch in the first few weeks after joining the study because viewing habits have been shown to change immediately after someone becomes involved in such research. It is therefore important to allow users to acclimatize to the study and if possible use equipment that is not too evasive.

Another way of reducing potential bias is not to disclose which website or page you are interested in and let the user start the study on another website. You can then guide the user to the relevant website through a search result or link and allow them to browse the page before moving onto another page or website.

Below I have summarised market leading eye tracking solutions and have categorised them according to key features.

Emotion Sensing Eye Tracking Solutions:

 

 1. Affectiva: A cloud-based eye tracking and emotional engagement research solution that uses a standard webcam or device camera. Developed using the world’s largest emotion database of more than 4 million faces across 75 countries, Affectiva has established norms to benchmark ad effectiveness by geography and product category. Unilever for instance uses Affectiva to measure emotion analytics for over 600 products in 70 countries. The solution also integrates with voice of customer tools and works with any panel provider.

 

The Affdex SDK allows developers to create interactive and emotion-aware apps and digital experiences that adapt to the emotional response of a user. The SDK monitors and reports on emotional responses from facial expressions by analysing the user’s face via the device camera, a video or even a single image in real-time.

Pricing: Quotations are available on request.

 

2.EyeSee: An online eye tracking and facial coding platform that uses the respondent’s own web cam to deliver fast, cost-effective and global eye tracking insights. Just before and during the eye tracking and facial coding research they monitor environmental factors and behaviour (e.g. head pose and head movement), and will if necessary recalibrate to ensure the accuracy of the research.

The platform is able to display a range of stimuli from static images, video and live websites. It can also make static images scrollable and zoom in by clicking on certain areas. For live websites the solution can activate or deactivate links and create overlays.

The facial coding functionality uses an algorithm to process webcam images which automatically identifies 7 emotions: Happiness, Surprise, Confusion, Disgust, Fear, Sadness and Neutral.  The output from the algorithm is double checked by humans to validate the accuracy of the outputs.

Pricing: No prices are displayed on the website. Please contact EyeSee for a quote.

 

 

3. Sticky:This is the World’s first cloud-based self-serve biometric eye tracking and facial coding solution.  Sticky utilises the user’s webcam and as a result the solution brings the cost of eye tracking and emotional engagement research within the reach of most medium to large companies .  Further, results can be obtained within hours rather than weeks that it might take for a traditional eye tracking study.

The webcam eye tracking software uses multiple tracking algorithms to identify the face, key features, eyes, iris and movement in 3D. This generates highly accurate eye tracking data and can be used to optimise all forms of digital content including images, ads, websites, videos and emails. Sticky’s Eye Portal allows users to identify areas of interest (AOI) produces data on three key metrics.

• The Seen Metric measures the percentage of the audience that saw the AOI.
• Stickiness informs how long on average was spent on it.
• Salience shows the average time it took a viewer to first see the AOI.

The solution is used for a range of purposes by Stick’s clients including website design optimization, ad placement valuation, article layout readability and more. Sticky also integrates with online surveys and it includes a feature to include survey questions within the portal.

Pricing: The cost is dependent upon the number of studies that you plan to run. Prices tend to start from just $225 per visual for 100 users or $250 for 400 users. However, the cost can reduce significantly as you increase the number of studies you run.

 

Traditional Eye tracking User Studies:

 1. Attensee: This solution measures where people pay attention to a web page by simulating focal and peripheral vision. The software achieves this by asking respondents to explore a website screenshot that is in a blurred state. When presented with a situation of distorted perception this activates our attention processing and our decisions of where to look are based upon limited information about the colours and shapes of objects on a web page. The data is then translated into gaze paths visualizations and eye fixation heatmaps.

Pricing: Only $49 a month, plus 50 cents per respondent,  for an unlimited number of surveys and tests with results the same day.

 

Eye tracking software to use in your own studies:

1. Eyegaze: The Eyegaze System uses the pupil-centre/corneal-reflection method to track where the user is looking on the screen. An infrared-sensitive video camera, mounted beneath the monitor takes 60 pictures per second of the user’s eye.

A  LED light mounted in the centre of the camera’s lens illuminates the eye. The LED reflects a small amount of light off the surface of the use’s cornea and pupil which reflects off the retina. This causes the pupil to appear white and enhances the camera’s image of the pupil and makes it easier for the image processing functions to locate the centre of the pupil. The computer calculates where the user is looking on the screen based upon the relative position of the pupil centre and corneal reflection within the video image of the eye. This provides an average accuracy of within a quarter inch or better.

Before operating the application the system must go through a calibration process to learn several physiological properties of the user’s eye in order to accurately determine where they are gazing on the screen. This calibration process takes about 15 seconds and does not need to be repeated if the user leaves the testing environment and returns later.

 

 

 2. EyeGuide: A real-time, wearable, mobile eye tracking system designed to be used by almost anyone. The headset is designed to allow researchers to communicate with users during testing and because it has its own built-in operating system you can use it with any platform you wish, such as Apple iOS or Microsoft Windows.  You can connect to the tracker using any device supporting the common 802.11g wireless protocol to see what the user is viewing.

The battery and storage pack allows you to record video and eye tracking data for more than five hours before recharging or downloading the test results. This allows you to record anywhere the user happens to be, though you can connect directly to AC power for continuous usage. Because it uses a hot mirror to calibrate and track the user’s eye movements there is no camera to interfere with the user’s normal vision. The headset is also designed for a range of head-sizes – from 6.5 to 8 (US).

Software to help you test, observe and analyse eye tracking data is included with the mobile tracker. This includes the ability to generate heat maps, gaze plots, dynamic clusters and pre-defined areas of interest. Further you can control testing by toggling the area of interest so that data collection can be focussed dynamically. For instance you can switch from emphasizing a laptop screen to smart phone or tablet screen.

 

 

3. Gazepoint: This desktop solution uses a camera mounted beneath the screen to calculate where the user is looking. It is ideal for UX and developers to conduct user testing to understand where on a page attention is focussed and how people navigate content. The tool offers eye gaze recording, Thinkaloud voice and webcam recording, heat maps, dynamic areas of interest, mouse clicks, pupil diameter data and where users stop scrolling.

 

 

4. Glance by mirametrix: A non-evasive, portable eye tracking system that sits just below the computer screen. It calibrates almost instantly and runs on Windows OS. It looks like an Xbox Kinect and can be used wherever you need to conduct user testing.

 

5. iMotions:  Provide a comprehensive package of biometric research hardware (including remote and mobile kit), software solutions and training. They are authorised  re-sellers of best-in-class hardware solutions and offer facial expression eye-tracking through Affectiva.com. As well as remote eye tracking hardware iMotions sell eye tracking glasses, biometric sensors and PC specs.

Algorithm based tools that predict user focus and attention:

 

1. Eyequant:EyeQuant uses algorithms and machine learning to predict how visitors will see and perceive website pages.  The software generates instant heatmaps covering attention, hot spots, engaged visitor analysis, perception maps, regions of interest, visual clarity and new visitor analysis. You can either enter a URL or upload an image for a new design.

Analysis is based upon eye tracking research, user studies and online experiments to understand how people perceive web designs. Statistical analysis is employed to identify correlations between specific design characteristics and user behaviours.

Design characteristics that are identified as influencing user behaviour are then used as variables in predictive modelling and machine learning is employed to determine the optimal weight and combination of these characteristics. Predictions are regularly benchmarked against eye tracking and other user studies to measure the accuracy of the modelling.

 

2. AttentionWizardfrom SiteTuners: Offers instant attention heat maps for images uploaded to the site. A more basic service without the enhancements provided by Eyequant and it doesn’t allow evaluation of live pages. To use the service you need to upload an image file between 500 – 1600 pixels wide and 500 – 1200 pixels high. If it is outside this range it won’t work.

Thank you reading my post. If you found this useful please share with the social media icons on the page.

You can view my full Digital Marketing and Optimization Toolbox here.

To browse links to all my posts on one page please click here.

• About the author:  Neal provides digital optimisation consultancy services and has worked for  brands such as Deezer.com, Foxybingo.com, Very.co.uk and partypoker.com.  He identifies areas for improvement using a combination of approaches including web analytics, heuristic analysis, customer journey mapping, usability testing, and Voice of Customer feedback.

• Neal has had articles published on website optimisation on Usabilla.com  and as an ex-research and insight manager on the GreenBook Blog research website.  If you wish to contact Neal please send an email to neal.cole@conversion-uplift.co.uk. You can follow Neal on Twitter @northresearch, see his LinkedIn profile or connect on Facebook.