Welcome to this week’s Question of the Week — your weekly AI & testing discussion!
AI-powered test script generation is everywhere right now. But how useful is it really in practice?
This week’s question: Have you used AI (like Katalon AI, GitHub Copilot, or ChatGPT) to write test scripts? What worked, what didn’t, and would you recommend it?
Share your honest experience below — the good, the bad, and the unexpected!
How to participate: Reply with your answer. Upvote replies you find helpful. The most insightful answer gets featured in next week’s community digest!
it is useful and accelerates the productivity
it’s a productivity boost, but the test engineer still needs to own reliability
AI definitely speed up script creation, reduce boilerplate, and help with smarter locators and flaky test fixes. From Katalon point of view its in-built AI tools like Studio Assist – improves script quality, debugging and more, while Katalon AI assistant can generate scripts directly from Jira use cases, which is really is big jump for the No code test automation platform
It surely needs the manual effort but I’d definitely recommend it for productivity and faster automation scaling.
AI is becoming inevitable in both professional and private life: Just like operating systems (Windows, Android, iOS) became foundational, AI is now embedding itself into daily workflows and tools.
So not only to write test script but everywhere.
One of the best feature that currently is giving is ability to create agent in KS.
If you not customized the prompt.md file then pls take some time to get it customized to you need and you would be amazed by the outcome of the test scripts that it generates.
You will have fewer updates to make in the script.
Go and get your hands dirty if not already done.
we do use it its not that much efficient but provide basic script to get started with
may it doesn’t have enough context knowledge.
it is useful and speeds up the productivity.But again, there is a need to review the script manually even if all cases are generated correctly.
Here is my honest take on this:
For me, AI has absolutely helped speed up the process and acts as a true catalyst for our testing workflows.
What worked really well:
-
Crushing Boilerplate Code: It eliminates the tedious setup work instantly.
-
Test Data Generation: If I need to generate multiple complex combinations of data, it works amazingly well and saves a ton of manual brainstorming.
The challenges (and how to beat them):
I definitely still see the hallucination part kick in when you try to force the AI to solve complex scripts or end-to-end workflows all at once. It gets overwhelmed and starts making things up.
My workaround: Instead of asking it for the final product, it’s much better to tear down the problem and work in steps. By feeding it smaller, bite-sized tasks, the output quality becomes much more reliable.
Overall, it’s an incredible tool if you know how to break down your workflow for it!