Visual AI

Introduction

You can use the Visual AI capabilities of our platform to ensure every part of the UI is tested.

If something changes on the screen, you will be alerted in the test results.

This method is also known as Visual Testing or Screenshot Comparison.

You can visually verify your entire page or only certain elements.

Guide

You can use one of these 3 options:

1. Page Matches Screenshot
2. Full Page Matches Screenshot
3. Element Matches Screenshot

The difference between Page Matches Screenshot and Full Page Matches Screenshot is that the first option refers to the viewport, while the second option refers to entire DOM.

You can find those options in:

• Add Assertion → Visual Assertion
• Start IF Statement → Visual Condition

You must provide a Baseline Screenshot URL for those options.

How to obtain a Baseline Screenshot URL:

1. Add a Take Screenshot, Snipping Tool or Generate Full Page Screenshot step in your test.
2. Execute the test.
3. Get the URL of the screenshot from the Results section.

If the screenshots have different sizes, the larger one will be automatically resized to match the smaller one.

The logs will display the baseline screenshot, the current screenshot and a third screenshot that highlights the visual differences:

Each Visual AI step will also have an Accept New Baseline button in the Results section. That action will update the Baseline Screenshot from that step with the Current Screenshot.

Excluding Elements

You can exclude specific elements from a Visual AI comparison by providing a comma-separated list of CSS Selectors.

This is useful for parts of the page that change between executions and should be ignored, such as timestamps, ads, or carousels.

For example:

.cookie-banner, #live-chat, .ad-slot

Anything matching those selectors will be excluded from the screenshot comparison.

What is the Visual Tolerance?

The Visual Tolerance (%) from the Advanced Settings can be configured to manage the relationship between the percentage result and the assertion result.

For example, if you select 5%, your assertion will fail if the difference between the screenshots is more than 5%.

The Visual Tolerance (%) doesn't influence the precision of the screenshot comparison and the percentage result.

It only determines the percentage at which the assertion will be marked as Passed or Failed.

What is the Visual Fuzz?

The Visual Fuzz (%) from the Advanced Settings can be configured to set the precision of the screenshot comparison.

A smaller value will lead to a more precise screenshot comparison.

A higher value will lead to a less precise screenshot comparison.

An optimal value for the Visual Fuzz (%) is somewhere between 2% and 20%, but you can select values between 1% and 50%.

The Full Page Screenshot option is only available for Web Tests.

Since the webdrivers only allow you to take a screenshot of the visible viewport, the Full Page Screenshots are generated by taking multiple screenshots and stitching them together.

This may cause some elements with the CSS property position: fixed to be displayed multiple times.

If you're testing in multiple browsers and resolutions, we recommend having baseline screenshots for each combination.

You can manage those combinations by using If Statements and System Variables.