I have started trying out the visual testing and Im impressed so far, thank you for your efforts. I still need more clarification please. When I run a different TestSuite with takeFullPageScreenshotAsCheckpoint(), this TestSuite is being compared to previous Baselines which don’t exist here.
Is there a way to separate each TestSuite’s Baselines so Katalon knows which screenshots to expect in each TestSuite?
I’ve updated the tutorial for Katalon Studio 7.7. I hope it would be helpful for you. If not, could you please give us more details (a piece of sample code would help a lot)?
I have multiple test cases for multiple screens which requires different visual testing of course. I called:
WebUI.takeFullPageScreenshot(RunConfiguration.getReportFolder() + ‘/keyes-search.png’)
in the Search screen and saved the initial screenshot as my baseline.
However, if I execute the other test case independently and take a screenshot at the Search Result screen:
WebUI.takeFullPageScreenshot(RunConfiguration.getReportFolder() + ‘/keyes-searchResult.png’)
It shows that a screenshot is missing “keyes-search.png”, but I am of no need of the first screenshot.
Is there a way to separate the screenshots’. For example, a folder for the set of screenshots of Search screen and separate screenshots for the Search Result screen instead of having them in the same baseline?
Hi @Tarek,
So you need a way to organizing your screenshots to a set of baseline collections and define which collection is used when run execution. Instead of comparing all baseline screenshots.
Yes, that is exactly what I’m trying to explain, sorry if my wording wasn’t correct. Is it available right now with the current release? Or would it be a future feature perhaps?
Thank you for the consideration! I’m looking forward for this project. Can I please add one idea too: allow the user to delete/rename Baseline screenshots.
How do you delete baseline images? I see no option to delete. Also Katalon seems to add duplicate images to the baseline when there is a mismatch rather than “know” that some elements on the same page have failed. It other words, it creates a new baseline when it should not.
Katalon seems to think that mismatches on a page are new page. So rather than compare against the existing baseline, it creates a new baseline as if it were a new page added.
You can “re-baseline” image if you tick the green checkmark as correct, refresh the page and button save to baseline will be enabled, this will “re-baseline” your image It’s a workaround for the delete of baseline images.
If you see one that says “missing” and you tick the green checkmark and click save to baseline it will “delete” it
@kevin.mcandrew1 The Keywords for Mobile are available now with Katalon Studio 7.8.0, and they can ignore regions when capturing screenshots. I added them to the first post.
So far looking good. The takeScreenshotAsCheckpoint command works well - no false positives so far (not sure if you have built-in tolerances or whether our app is pixel perfect each time?).
I didn’t try takeAreaScreenshotAsCheckpoint (not sure if I understand the rectangle aspect), and takeElementScreenshotAsCheckpoint didn’t work for me - the image it captured did not resemble the element I’d identified (I tried three separate objects).
I also couldn’t see how to exclude regions? Unless you mean by being able to capture an element or area that they are in effect excluding other areas?
Please can I make a couple of feature requests and report a possible small bug…
Possible bug: in the TestOps Visual Testing dashboard, within an execution, the ‘Save to Baseline’ button is not immediately enabled after accepting a new or changed image…as a workaround, refreshing the page enables this button.
New Feature Request #1: Would it be possible to take ‘full page’ screenshots (i.e. the full height, where a vertically-scrollable page is displayed) please?
New Feature Request #2: For excluding regions, would it be possible to take a screenshot of the whole app (either full-height, or as it is now), and be able to exclude specified element(s)? For example, in the following image (just a random example), the pink area might be some sort of container element for dynamic content that could change with every login? (obviously you may have covered such a scenario already?)
Hi there,
Love this feature, think it has lots of potential and so far, for my minor use case (verifying a blob pdf view) it works great (Version 7.8.1).
However, 2 questions:
In my test I want to verify twice against the same baseline. So I have in my test check 1:
Question two: If the screenshot does not match the baseline the test case itself will still pass. I presume because the actual verification happens only AFTER the test is finished and uploaded to the TestOps…
So we will have no failure in the test management executions, but we will have to monitor this in the visual testing executions. Isn’t there a way to make the test case fail in the test management executions, or make it at least visual over there?