I have made a demo project at
This project proposes a set of Custom Keywords which enable you to test arbitrary HTML elements visually by taking and comparing screenshots in various ways.
Problem to solve
What is built-in
Katalon Studio provides a limited number of built-in keywords for testing web pages visually.
Shortages
I would enumerate shortages of the built-in keywords above.
-
WebUI.takeScreenshot(File)
enables you to take screenshot of viewport of the target web page, but it does not allow you to take screenshot of a particular HTML element of your interest. -
WebUI.verifyImagePresent(TestObject expected)
requires you to prepare a image file as the expected image before running test. But Katalon Studio does not provide any method for you to prepare the expected image out of the target web page. Katalon just assumes you can find the expected image file somewhere. -
WebUI.verifyImagePresent(TestObject expected)
tries to find the expected image anywhere one or more times in the target web page. You can not specify which HTML element to compare against the expected image.
Solution proposed
aShot, WebDriver Screenshot utility, enables you to take screenshot of particular HTML element. Also it provides images comparison. My Custom Keyword com.kazurayam.ksbackyard.ScreenshotDriver
wraps aShot so that you can utilize aShot features in Katalon Studio easily.
Custom Keywords
ScreenshotDriver#saveElementImage
Description
Takes screenshot of a HTML element in the target web page and store into file
Parameters
Param | Param Type | Mandatory | Description |
---|---|---|---|
element | TestObject | Required | target HTML element to take screenshot of |
file | File | Required | File into which PNG image is saved into |
Example
// save screenshot of Google Logo area
TestObject logoArea = findTestObject('Page_Google/div_logoArea')
File file1 = workdir.resolve('logoArea.png').toFile()
CustomKeywords.'com.kazurayam.ksbackyard.ScreenshotDriver.saveElementImage'(logoArea, file1)
ScreenshotDriver#verifyImagesAreSimilar(File, TestObject, …)
Description
Compare actual screenshot of a HTML element in the target web page against the expected PNG file. Difference of 2 images are calculated. If the difference is less than or equal to the given criteria, then verifies images are regarded similar.
Parameters
Param | Param Type | Mandatory | Description |
---|---|---|---|
expected | File | Required | PNG File to compare HTML element against |
actual | TestObject | Required | target HTML element in the target web page |
criteriaPercent | Double | Required | e.g. 5.0 percent |
snapshotDir | File | Optional | default to /tmp directory. When FAILED, 3 PNG files will be saved into this directory. These are the snapshots of the expected image, the actual image, and the difference image. The snapshots will NOT be saved when PASSED as default. But you can toggle it ON by calling ScreenshotDriver.setForceSnapshots(true)
|
flowControl | FailureHandling | Optional | default to CONTINUE_ON_FAILURE |
Example
TestObject logoArea = findTestObject('Object Repository/Page_Google/div_logoArea')
File file1 = datadir.resolve('google_logo.png').toFile()
CustomKeywords.'com.kazurayam.ksbackyard.ScreenshotDriver.verifyImagesAreSimilar'(
file1,
logoArea,
3.0,
workdir.resolve('a').toFile(),
FailureHandling.CONTINUE_ON_FAILURE)
Or you can simplify the above code as follows:
...
CustomKeywords.'com.kazurayam.ksbackyard.ScreenshotDriver.verifyImagesAreSimilar'(
file1,
logoArea,
3.0)
There are a few more keywords supported. see the GitHub project for detail
How to run the demo
- download the zip file of this project from Realses page
- unzip it
- open the project with your Katalon studio
and then
4. TC1_saveElementImage
Open Test Cases/TC1_saveElementImage
and run it. When finished, you will find the screenshot of
tmp/TC1
directory.
5. TC2_verifyImagesAreSimilar_or_Different(File,TestObject)
Open Test Cases/TC2_verifyImagesAreSimilar_or_different(File,TestObject)
and run it. This test case will fail. The log was as follows:
11-01-2018 04:42:58 PM - [FAILED] - Test Cases/TC2_verifyImagesAreSimilar_or_Different(File,TestObject) FAILED because (of) (Stack trace: com.kms.katalon.core.exception.StepFailedException: images are expected to be similar but are different, difference=99.35%, snapshots were saved in C:\Users\qcq0264\katalon-workspace\ComparingScreenshotsOfWebElement\tmp\TC2\b at com.kms.katalon.core.util.KeywordUtil.markFailed(KeywordUtil.java:15) at com.kms.katalon.core.util.KeywordUtil$markFailed.call(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:125) at com.kazurayam.ksbackyard.Assert.stepFailed(Assert.groovy:68) at com.kazurayam.ksbackyard.Assert.assertTrue(Assert.groovy:27) at com.kazurayam.ksbackyard.Assert$assertTrue.call(Unknown Source) at com.kazurayam.ksbackyard.ScreenshotDriver.verifyImagesAreSimilar(ScreenshotDriver.groovy:316) ...
You can find the reason why it failed by looking at the snapshots. See the following snapshots.
The expected image is tmp/TC2/b/verifyImagesAreSimilar(File,TestObject).expected.png
The actual image is tmp/TC2/b/verifyImagesAreSimilar(File,TestObject).actual.png
The difference image is tmp/TC2/b/verifyImagesAreSimilar(File,TestObject).diff(99.35).png
The expected image and actual image is NOT similar, therefore the test case TC2 fails.