Dynamic test suites doesn't find tagged test cases

Bug Report
1

After tagging a test case, it cannot be found in a Dynamic test suite.

KSE v 10.4.1 on windows

Steps:

  1. open test case
  2. click Properties tab
  3. in Tag input, add the word BLOCKED and save tc
  4. create new dynamic test suite
  5. go to ‘query builder’ button
  6. add word BLOCKED to tag and then click Search
  7. note that query now reads tag=(BLOCKED)
  8. click Preview button - instead of one result, nothing happens
  9. observe that using the advanced search instead with word BLOCKED returns result for the tc as expected, but not the dynamic test suite
1 Like
2

@Dan_Bown
The issue occurs because Dynamic Test Suites in Katalon rely on a cached index of test case metadata, which doesn’t always refresh immediately after adding tags. That’s why Advanced Search works (it reads live data), but Dynamic Test Suite preview returns no results. The problem is not with your tagging but with indexing. To fix it, try reopening the project, re-saving the test case, or re-adding the tag. Also ensure simple tag formatting (e.g., lowercase). In some cases, this is a version-specific limitation, so upgrading Katalon or refreshing the project cache usually resolves the issue.

What’s going wrong

Dynamic Test Suites rely on an internal index/cache of test case metadata (including tags). Sometimes, when you add or modify tags, that index doesn’t refresh immediately. That’s why:

  • Advanced Search works (it reads directly)

  • Dynamic Test Suite doesn’t return results (it depends on cached indexing)

What you can try (fix/workarounds)

  1. Refresh/Rebuild project

    • Close Katalon

    • Reopen project
      OR

    • Go to Project → Clean up (if available)

  2. Re-save the test case

    • Open the tagged test case

    • Remove the tag → Save

    • Add it again → Save

  3. Check tag formatting

    • Avoid spaces or special characters

    • Try simple lowercase like: blocked instead of BLOCKED

  4. Use “contains” instead of exact match

    • In Query Builder, try a different operator if available

  5. Check for version bug

    • This behavior has been reported in some versions

    • If possible, try upgrading or switching version to confirm

3

What Actually is the bug.

The exact reproduction on the Katalon forum is: open a test case → click Properties tab → add the word BLOCKED to the Tag input and save → create a new Dynamic Test Suite → open Query Builder → add BLOCKED to the tag field and click Search (query becomes tag=(BLOCKED)) → click Preview — instead of returning one result, nothing happens. However, using Advanced Search with the word BLOCKED returns the result for the test case as expected.

Root Cause 1: Stale internal index — v10.4.1 regression

The Built-in Query Provider reads test case metadata from an in-memory index built when the project loads. In v10.4.1 on Windows, saving a tag does not trigger a re-index of that file. The query runs against stale data and returns zero results silently. Advanced Search reads the .tc files directly from disk, bypassing the index, which is why it still works.

Root Cause 2:Tag not flushed to disk before query runs

If the test case editor tab is still open when Preview is clicked, Windows may not have fully written the .tc XML file yet. Even a correctly saved tag can appear absent if the file write is buffered. Always close the test case tab after saving before running the DTS query.

Root Cause 3:Case mismatch or whitespace in tag value

Tag matching is case-sensitive in all Katalon versions. BLOCKED, blocked, and Blocked are three different tags. A trailing space typed in the Properties tab also prevents a match. The query box value and the Properties tab value must be identical character-for-character.

Root Cause 4:No query provider active

If the Query Provider dropdown reads “No query provider available”, the search engine has nothing to execute against. From v8.5.0 this is built-in; on older installs or a clean re-install the plugin may need to be reloaded via Account → Reload Plugins after licence activation.

How to fix it(Steps on How)

Step 1:

Verify the tag is saved. Open the test case → Properties tab → confirm the tag → press Ctrl+S → close the tab and reopen it. Then open the .tc file in Notepad and confirm <tag>BLOCKED</tag> is present.

Step 2:

Confirm the query provider is active. After installing the Basic Search For Dynamic Test Suite plugin, the query syntax standard becomes “Built-in”. BrowserStack If the Dynamic Test Suite still shows “No query provider available,” go to Account → Reload Plugins.

Step 3:

Force a project re-index if on v10.4.1. Go to File → Close Project → reopen the project. The index is fully rebuilt on load. Wait for the status bar to idle, then try Preview again.

Step 4:

Switch to Advanced Tag Search. After activating the Test Case Management with Tags plugin, the Query Provider is set to Advanced Tag Search. Katalon This provider reads directly from the file system and bypasses the stale index entirely.

Step 5:

Verify exact syntax. Type tag=(BLOCKED) directly into the query box. The value inside the parentheses must match the tag exactly — same case, no surrounding spaces.

Step 6

For KRE/CI: add the parameter -testSuiteQuery to your generated KRE command, for example -testSuiteQuery="tag=(BLOCKED)" GitHub, combined with your -apiKey so the plugin is authenticated.

Step 7:

If nothing works: upgrade KSE. The v10.4.1 regression is resolved in later releases available at Katalon Studio Download | Katalon.

Solutions

Option 1:

The fastest solution with no reinstall required is to change the Query Provider dropdown in the Dynamic Test Suite from “Built-in Search” to Advanced Tag Search, then type tag=(BLOCKED) directly into the Query box and click Preview. After activating the Test Case Management with Tags plugin, the Query Provider is set to Advanced Tag Search. GitHub This provider reads from the file system rather than the index, and is confirmed working in v10.4.1.

Option 2:

To force a re-index: close Katalon and reopen the project, or go to Project → Clean up if available. Katalon The index is fully rebuilt on every project load.

Best Option

The permanent fix is upgrading KSE. The v10.x release cycle has had active bug fixing across the platform Katalon, and the index regression introduced in 10.4.1 is not present in subsequent releases available at Katalon Studio Download | Katalon.

Dynamic
Dynamic357×520 50.6 KB

4

thanks for the feedback, I’ll try this solution but won’t be immediately, The documentation for tagging could be clearer but I’ll assume a variety of logic symbols can be used for dynamic test suites

5

yep, this is a known behavior in Katalon Studio, not your mistake.

Root cause (short):
Dynamic Test Suite uses cached index, not live data → your new tag isn’t picked up yet.
Advanced Search works because it reads live metadata.

What actually works (quick fixes):

  • Close & reopen project (most reliable)
  • Re-save the test case (remove tag → save → add again → save)
  • Try lowercase tag (blocked instead of BLOCKED)
  • Recreate the Dynamic Test Suite (forces re-index)

If still broken:

  • It’s likely version issue (10.4.1 has reports like this)
    → try upgrade to confirm

:backhand_index_pointing_right: TL;DR: nothing wrong with your steps, it’s indexing lag. Restart/reindex usually fixes it