I have created a project at
Please refer to this Github repository for running codes and description in detail.
----
Problem to solve
What do I mean by the word Visutal Testing? Any toolset that provides a solution to my requirements as follows, I would regard it a Visual Testing solution.
- Often I have to look at the pages of a Web site. Maybe I am a developer, a designer, a tester or just a fan of the site.
- I want to look at as many pages of the site as possible. Wide coverage matters.
- I want to compare the view of 2 environments; e.g. Production and Development.
- I want to take full-page screen shots of all pages as evidence
- After taking screenshots, I want to check them to find out if there are any differences in view between the two.
If the target Web site has 100 pages, it would take me more than 60 minutes to run through. Itās too tiring, too boring. I donāt like doing such a bullshit job! Therefore I would add the final term:
- I want to automate it.
I just want to be notified if 2 environments have some significant visual differences. I would not require the tool to be equipped with full fledged debugging functionalities. Preferably it should be free of charge.
Solution
As for the file path problem, I have developed another project titled:
This project demonstrates how to take multiple sets of web page screen shots and store them into a well-structured file system tree. This demo project uses another GitHub repository:
This project provides a jar file which implements MaterialRepository
= well-defined file tree and provides access methods.
As for the full-page screenshot problem, I found that the library aShot
solves like a charm. I wrote a report how I utilized aShot in Katalon Studio:
I was impressed that aShot provides ImageDiff. This utility class enables me to check very easily how much differences there are amongst 2 images. I wanted to use ImageDiff
in Katalon Studio. So I have developed another GibHub repository:
In there I have developed Katalon Custume Keyword com.kazurayam.ksbackyard.ScreenshotDriver. ScreenshotDriver is just a small wrapper for aShot. This makes it a bit easier to use aShot functionality in Katalon Test Cases.
I have integrated those external resources to build a toolset of āVisual Testingā in Katalon Studio.
------------------
Input
A Test Suite Collection Test Suites/Execute_twins
executes a Test Suite called Test Suites/twins_capture
twice. Each execution takes screenshots of the following URLs:
- http://demoaut.katalon.com/ --- called Production environment
- http://demoaut-mimic.kazurayam.com/ --- called Development environment
Once screenshots are taken and stored in PNG files, Test Suites/Execute_twins
executes a Test Suite called Test Suites/twins_exam
. This test suite compares each pairs of screenshots to find if any significant visual differences are there. It generates a HTML file to show screenshots with the assertion result.
Output
- Example Materials/index.html in action: VisualTestingInKatalonStudio/Materials
You should open index.html with your browser by double-clicking the local file.
index.html
The test suite collection generates ./Materials/index.html
. This HTML shows a list of source images plus the images as comparison result.
File path: <projectDir>/Materials/index.html
ImageDiff
If you click the line with purple background color, you will see a ImageDiff with a lot of red-portion. The red portion shows the differences between the two source images.
Production page
Development page
Twins-mode and Chronos-mode
Initially VisualTestingInKatalonStudio supported Twins mode which compares a pair of URLs of Application-Under-Test at a given time. This aimed to verify if a development environment is identical to a production environment of my AUT.
Later in the version 1.10.0, VisualTestingInKatalonStudio supported Chronos mode which compares the current set of screenshots against the previous set of screenshots of a Web site. The Chronos mode enables me (and you) to check status of the AUT at different timing: before/after any system changes ā application software upgrades, changes to configs, OS patches, network reconfiguration, database migration, etc.
How to enable your own Katalon project to perform āVisual Testingā
See the following documentation: