I was happy to find that Accessibility testing was considered by Katalon with official documentaion such as: Accessibility test automation in Katalon Studio | Katalon Docs and pointing to the well known axe-core library. This topic was debated on this forum in the past but never got to any conclusion / fruitful guide for newcomers to follow so I am starting this new thread here, especially knowing we recently moved to Katalon 10.x which can have some consequences…
I happily followed the link in the Katalon official doc, downloaded the recommended version of package selenium-4.4.1.jar then added it to my KSE and tried to create a keyword as suggested, then ran a simple test and… it failed.
I then removed the fancy / unecessary keyword complexity (can come as step 2 / enhancement for accessibility testing scale up) and came with this simple root script to check axe core capabilities fast:
import com.deque.html.axecore.selenium.AxeBuilder as AxeBuilder
import com.kms.katalon.core.webui.driver.DriverFactory as DriverFactory
WebUI.openBrowser('')
WebUI.navigateToUrl(targetURL)
AxeBuilder axe = new AxeBuilder()
def driver = DriverFactory.getWebDriver()
axe.analyze(driver)
I get a similar error to when I was trying to run via keyword, and I fear this is not working anymore with Katalon 4.x / Selenium 4
In the modern Selenium 4 and later versions, the setTimeout and related methods are used with the java.time.Duration class, moving away from the older, deprecated long time, TimeUnit unit signature. Therefore your application has some version incompatibility problem.
1) Do not override Selenium in Katalon; only add the axe JAR
Katalon’s own docs recommend adding only the axe-core Selenium integration JAR, not a Selenium JAR, and explicitly warn that the integration is “not officially supported” and version-sensitive.
Keep only the axe-core Selenium integration JAR from Deque (for example, axe-selenium-java/axe-core-selenium), so Katalon’s bundled Selenium remains the only Selenium implementation on the classpath.
Then retry a minimal script:
import com.deque.html.axecore.selenium.AxeBuilder
import com.kms.katalon.core.webui.driver.DriverFactory
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI
WebUI.openBrowser('')
WebUI.navigateToUrl(targetURL)
def driver = DriverFactory.getWebDriver()
new AxeBuilder().analyze(driver)
If the same NoSuchMethodError persists even without your own Selenium JAR, it means the Deque adapter itself is not compatible with the Selenium version inside Katalon 10.x.
2) Use the browser-injected axe.min.js pattern instead
Because axe-selenium-java is deprecated and tightly coupled to older Selenium APIs, a more robust approach is to inject axe.min.js directly and call it via executeAsyncScript, avoiding the AxeBuilder Java wrapper entirely.
High-level idea:
Download axe.min.js from Deque and place it under, for example, Include/scripts/axe/axe.min.js in your project.
In a custom keyword:
Read the file contents into a String.
driver.executeScript(axeScript) to inject axe into the page.
This issue suggests that even the latest version of the Axe product has the aforementioned method signature problem outstanding.
I suppose that @yann.sautreuil needs to wait for them to fix the problem. Or you might want to fix the problem in the Axe project and contribute a pull-request to them.
By the way, Selenium-Java 4.0.0 was released at Oct 2021, 4 years ago. This means the Axe project was not aware of the method signature incompatibility problem for 4 years. Maybe nobody has noticed the issue up to now when @yann.sautreuil found it. Presumably the issue does not matter for the Axe users very much. I am afraid that the Axe project would not work on this issue after the 4 years of ignorance.
Indeed @kazurayam I contacted the project owner yesterday after you confirmed my thoughts about selenium 4 compatibility and they created this issue. So happy to see things moving so fast. Need to wait now indeed.
Also where do you see that it’s not officially supported, the katalon page seems on the contrary to say it works very well with Katalon and is the go to way of doing accessibility tests: Accessibility Testing: A Complete Guide - “You can easily automate accessibility testing at scale by combining Katalon’s low-code automation features with Axe’s accessibility testing tool thanks to the Axe-core library integrated in Katalon Studio.”
Anyway when I mentioned my error to the repo owner they replied by creating a bug on their side, recognizing there is something they need to adjust.
Regarding the second option, I spent half an hour fighting to run the project locally and generate this simple js file with npm run build and stuff, throwing errors in my face and requesting to install X so that I can then install or run Y and so that I can… I was already fedup with this nonsense 20 years ago, I still am. Now if someone has a better configured machine / dev environment to generate the js file, I would be happy to test your second approach.
I guess that Accessibility tests would be more intuitively implemented in JavaScript/TypeScript locally on browser than via the remote WebDriver protocol in Java/Groovy. So Playwright would be better fit than Selenium for Accessibility Testing.
Why not you try switching from Katalon to Playwright? To try it, you do not have to wait for the axe-core-maven-html project to improve itself for Selenium.
