Katalon’s AI-powered solutions help QA teams by automating intelligent test generation, execution, and analysis across APIs, cloud services, Big Data, and analytics platforms, ensuring faster validation and reduced risk in complex enterprise ecosystems. Testers should evolve by strengthening skills in automation frameworks, API/cloud testing, and data-driven validation, while embracing AI-assisted tools to shift from manual checks to strategic quality engineering that supports digital transformation.
this is a different ball game , cant be done so easily with just some scripts or utilities
Yes, hmm that I understand
Really?
Do we really provide all code base to AI?
Good question, Akshay — and no, you don’t need to feed your entire codebase into an AI tool for this to work. That’s a common misconception, so glad you raised it.
What’s actually happening in “AI predicts risk areas” is usually one of two things, and neither requires exposing your full soource:
1. Metadata-based risk models (most common approach)
These models don’t read your code line-by-line — they train on metadata about your codebase history:
- File/module change frequency (churn) from git logs
- Which files were touched in commits linked to bug-fix tickets (via Jira/commit message correlation)
- Code complexity metrics (cyclomatic complexity, file size, number of dependencies)
- Historical defect density per module
This is basically statistics on your commit and defect history, not the code itself. Tools like Launchable and some CI risk-scoring plugins work this way — they ingest commit metadata + test results + Jira links, not your source tree.
2. Static analysis + local/on-prem models
For deeper code-level risk analysis (e.g. flagging risky diffs), teams either:
- Run static analysis locally and only send derived signals (complexity scores, diff size, affected function names) to the AI layer, or
- Use a private/on-prem LLM deployment (or an enterprise-tier tool with a data-processing agreement) if they genuinely want code-level review — which is a very different trust boundary than pasting code into a public chatbot.
So Practically : if your org is doing this with a SaaS AI tool, check what data leaves your network — for most risk-prediction use cases it’s commit metadata and defect history, not raw code.
The metadata-based approach makes sense for identifying potentially risky modules.
But how does the AI determine (just from the metadata)
- Which specific functionality is impacted by this new change?
- Whether the change affects existing behavior, introduces new behavior, or is simply a refactoring?
- Which specific set of tests to re run?
- Which specific set of regression tests to run?
I’ll explain this based on my understanding of how these systems typically work.
To make it easier I have taken a generic example. So here it is not purely AI driven but much of it relies on deterministic engineering principles.
1. How does the system decide which tests to rerun?
This is generally not an AI enabled. Instead, it is handled through Test Impact Analysis (TIA).
Each time a test suite executes, coverage tools like Jacoco and others record which tests interact with which parts of the codebase. It tracks exactly which lines, methods, and branches (such as if/else statements) are executed during a test run. ANd gradually this creates a traceability map, for example: Test_X covers Functions A, B, and C.
When a code change is introduced, the system analyzes the modified functions and consults this mapping to identify the tests associated with those areas. It then reruns only the relevant tests rather than executing the entire suite.
Most implementations also consider dependency chains. For example, if Function A depends on Function B, and Function B is modified, tests covering Function A may also be included, even if they do not directly interact with Function B.
So that’s why I say that this process is largely deterministic: identify changed code, map it to impacted tests, and execute those tests. In the tools like Launchable it is using the same. And now the AI component is typically used to prioritize or rank tests based on risk, rather than determine the impacted tests themselves.
- How are additional regression tests selected?
Here, it is based on the historic data like whenever Module X experiences failures, Module Y is also affected approximately 30% of the time. Based on such historical trends, the system may recommend or include additional regression tests from related areas, even when there is no explicit code-level dependency between them. So here unlike above Test Impact Analysis, the deterministic thing, it is based on statistical inference and historical behavior patterns.
So AI can only assist with impact estimation and risk assessment, but human judgment is still critical when interpreting the true intent and significance of code changes.
Hi @jahnavi.b, I’d say the most practical skill is learning how to work with AI throughout an actual testing workflow, not just learning about AI or collecting prompting techniques.
A few skills I would focus on:
Giving AI the right context: requirements, business rules, application behavior, test data, and expected outcomes.
Reviewing AI-generated tests critically: checking coverage, assertions, edge cases, and whether the tests address real business risks.
Testing and refining AI output: run what AI creates, identify what does not work, and use that evidence to improve the result.
Using AI beyond test generation: for requirement analysis, test data creation, automation, failure investigation, and reporting.
Maintaining testing and technical fundamentals: testers still need to understand APIs, locators, assertions, logs, and basic scripting to know whether AI’s suggestions make sense.
Deciding what is worth testing and automating: AI can generate many test cases, but testers still need to identify which ones actually matter.
The best way to build these skills is to start with one real task. For example, use AI to generate tests from a requirement, review the output, run the tests, and note what context helped improve the result.
I second this