I appreciate that this (the subject above) may seem a simplistic question. And some will say “just use the visual test feature”, but having read the docs I don’t sense this “simple” use case is within the intended purpose of that feature.
It doesn’t seem there’s a simple feature within that which can be enabled to “capture all the content of each visited page”, and then “flag during a test if the content differs from what was captured”.
Is there anyone who has done such simple regression testing using that feature?
I do realize, of course, that a classic dilemma with such “simple regression testing” is that “some unimportant part of the page will change”. I’d hope that there’s provision to somehow mark that to be ignored–preferably without need of coding, but I’d accept if that was the only option.
And if such accommodation of a “detected but unimportant change” could be accomplished based on assessing the RESULT of a test run (rather than being anticipated in advance during the CREATION of the test), that would be all the better.
But again I’d be happy if it ONLY detected and flagged that the content had changed, even if I’d have to somehow rebuild the test to ignore that change. Anything would be better than doing it by hand, as so many try to do.
If the conclusion is that indeed “this is not something Katalon can handle readily”, I would welcome hearing from anyone who might propose any other solution that can handle it. Thanks.
Do you want a solution that automatically visits all pages that comprise with your web site?
I believe that all of the webui testing automation solutions in the marketplace expect users to develop a set of custom codes that enumerate URL to be tested and navigate to visit the URLs , then do something on the visited pages (take screenshots, etc). None of them can guess what you want to do without your coding.
Do you want the solution to automatically find and enumerating the URLs without asking you which to visit?
In other words, is it acceptable for you to author a set of codes that visit the your target pages for yourself, not automatically? If you want to check 100 diffierent views of a site, will you be prepared to develop a set of 100 codes?
Above line shows a case where the “UI/logo” element will be ignored.
Obviously, this requires your coding effort. It requires you to identify which HTML element in the page to be ignored. It requires you to write an appropriate TestObject that select the HTML element to be ignored.
I hope you mean “satisfy that specific aspect”, rather than me. I’d be satisfied if all the other aspects were covered by Katalon.
As for suggesting I ask them, are you indicating that these forums are not a way to cause them to be made aware of such a feature request? Or is there a better way? Again, I appreciate your chiming with the thoughts so far.
That would be a reasonable mod, sure. What’s not clear is if the recorder has provision to automatically create the call to takeFullPageScreenshot in the first place–including coming up with a unique name for the screenshot for each page, and indeed a unique folder holding those for each test.
I’d think this use case is common enough that it would be provided for…yet my reading of the docs and forum left me asking the question–and I suppose I should have clarified that in my original post here.
Thanks. To be clear, I am not seeking it to auto check all pages that comprise my site. I’m happy to use the recorder to visit a subset of them in a given sequence.
That said, I am seeking to regression-test the content of each in that sequence. And my question is simply a) whether that’s a feature at all and b) if it can be done without coding, or with only slight modification to the script created by the recorder.
In my humble opinion, the WebUI.takeFullPageScreenshot keyword and the Visual Testing feature of Katalon Studio are poorly designed.
How poorly? — The system asks you (users, testers) to invent a custom code that creates a unique folder tree holding the screenshot images, and it asks you to give unique/useful name to each PNG files. Katalon provides no support to manage a storage for the screenshot PNG files.
Is it a trivia? — No, it isn’t. I think it is a big technical issue.
I have ever struggled with this. I have developed a library named “materialstore” which is
A domain-specific file system to store “materials” (screenshots, HTML, JSON, XML) collected during End-to-End testings using Selenium WebDriver etc. Features to make “diff” and compiling HTML reports are also included.
On top of the materialstore library, I developed a demonstration project for Katalon Studio. You can find a post about it here:
In order to implement this, I reinvented a lot of wheels. I developed a “custom keyword” to take screenshot and save into a “store” filesystem. I developed a query engine over the “store” filesystem. I developed functions for image comparison. I developed a report where you can see the original pair of screenshots and the diff image. In the end, I made my code entirely independent of Katalon Studio. See inspectus4selenium-sample-project which runs with Gradle+Selenium4+JUnit plus my libraries without Katalon Studio.
I devoted many months to develop this libary. It was a difficult task. Katalon Studio just lacks such capability.
So I feel an empathy with @charlie who raised this topic.
Thanks. And I recall now I’d found that entire thread and github repo before asking here. (Of course, I’d not connected the dot that you were the same person.)
So yep, it was dismaying to see the hoops you’d had to jump through. I was really hoping things might have improved since then. Alas, it seems not.
I do appreciate the commiseration, though I really hope someone in a position of authority might throw us a bone, especially if indeed either we’re still missing something, or perhaps some feature may be under consideration.
I am not a paying customer, and as a new one seeking thus functionality as my primary use case, I’d not be inclined to become one.
Again, I raised the issue as I thought it to be a rather common use case. The lack of support/lack of demand could be a chicken/egg situation, I suppose.
We’ll see over time if anyone else jumps in with thoughts.
Thanks @charlie and @kazurayam for the discussions. This is helpful for us to understand the expected use cases and how our products should cater the needs of variety of users. All ideas are welcome.
P/s: we are having a new ideaExchange portal where users can share their ideas and our team will provide validation for future product development. With that being said, I believe we will have a good place to gather ideas from users.