I have multiple test cases for multiple screens which requires different visual testing of course. I called:
WebUI.takeFullPageScreenshot(RunConfiguration.getReportFolder() + ‘/keyes-search.png’)
in the Search screen and saved the initial screenshot as my baseline.
However, if I execute the other test case independently and take a screenshot at the Search Result screen:
WebUI.takeFullPageScreenshot(RunConfiguration.getReportFolder() + ‘/keyes-searchResult.png’)
It shows that a screenshot is missing “keyes-search.png”, but I am of no need of the first screenshot.
Is there a way to separate the screenshots’. For example, a folder for the set of screenshots of Search screen and separate screenshots for the Search Result screen instead of having them in the same baseline?
How do you delete baseline images? I see no option to delete. Also Katalon seems to add duplicate images to the baseline when there is a mismatch rather than “know” that some elements on the same page have failed. It other words, it creates a new baseline when it should not.
You can “re-baseline” image if you tick the green checkmark as correct, refresh the page and button save to baseline will be enabled, this will “re-baseline” your image It’s a workaround for the delete of baseline images.
If you see one that says “missing” and you tick the green checkmark and click save to baseline it will “delete” it
So far looking good. The takeScreenshotAsCheckpoint command works well - no false positives so far (not sure if you have built-in tolerances or whether our app is pixel perfect each time?).
I didn’t try takeAreaScreenshotAsCheckpoint (not sure if I understand the rectangle aspect), and takeElementScreenshotAsCheckpoint didn’t work for me - the image it captured did not resemble the element I’d identified (I tried three separate objects).
I also couldn’t see how to exclude regions? Unless you mean by being able to capture an element or area that they are in effect excluding other areas?
Please can I make a couple of feature requests and report a possible small bug…
Possible bug: in the TestOps Visual Testing dashboard, within an execution, the ‘Save to Baseline’ button is not immediately enabled after accepting a new or changed image…as a workaround, refreshing the page enables this button.
New Feature Request #1: Would it be possible to take ‘full page’ screenshots (i.e. the full height, where a vertically-scrollable page is displayed) please?
New Feature Request #2: For excluding regions, would it be possible to take a screenshot of the whole app (either full-height, or as it is now), and be able to exclude specified element(s)? For example, in the following image (just a random example), the pink area might be some sort of container element for dynamic content that could change with every login? (obviously you may have covered such a scenario already?)
Question two: If the screenshot does not match the baseline the test case itself will still pass. I presume because the actual verification happens only AFTER the test is finished and uploaded to the TestOps…
So we will have no failure in the test management executions, but we will have to monitor this in the visual testing executions. Isn’t there a way to make the test case fail in the test management executions, or make it at least visual over there?
Thank you for the feature requests, I think they are very helpful. We will add them to our backlog.
Regarding you question, you can pass a list of redacted objects, e.g. Mobile.takeScreenshotAsCheckpoint('checkpoint_name', [findTestObject('Application/android.widget.TextView - App')]). The feature currently does not work as expected on some Android models.
In the future, we will allow specifying redacted regions on Katalon TestOps UI (maybe by draw a rectangular shape on baseline images).