Welcome back to Question of the Week — the AI edition!
AI tools have come a long way, but there are still plenty of gaps. What’s the feature or capability that would genuinely change how you test if AI could do it?
This week’s question: What’s the #1 thing you wish AI could do in test automation that it still can’t do reliably?
Be specific — the more detail you share, the better the discussion!
How to participate: Reply below. Upvote answers you agree with. Best response gets featured in our weekly digest!
Right now, AI is incredibly good at syntax generation (e.g., “Write a Playwright script to click this button”). What it cannot do reliably is look at a complex, end-to-end user workflow and determine if the system actually behaved correctly from a business logic standpoint.
What AI can do today:
-
Identify selectors and element IDs.
-
Generate boilerplate test scripts.
-
Catch basic visual regressions (e.g., “this button turned blue instead of red”).
What AI fails at reliably:
Knowing if the data behind the UI makes sense after a complex series of actions.
The Dream Feature: “Intent-Driven Oracle Testing”
I wish I could give an AI a high-level business requirement, and have it independently verify the system’s state across multiple layers (UI, API, and Database) without me having to hardcode every single assertion.