Advice requested for object naming maintenance vs faster test case builds

I’m building out automation for my company from scratch. We currently have 233 test cases for regression testing an enterprise packaged application on a quarterly basis.
I’m torn between building the test cases quickly by accepting the naming convention spy web or katalon recorder gives versus spending the extra time up front to rename them to something more identifiable. This has added quite a bit of extra time.
I’m looking for feedback from Katalon users on how difficult test script maintenance becomes if you don’t re-name the objects on first build. I will have comments/descriptions for every step so I won’t need to look at the object to know what it is supposed to be clicking on.
Do you think the time savings up front by going with the “sloppy object name” route is negated during the test case maintenance stage?
Either way, I’m really banking on the self healing technology Katalon has developed to reduce our script maintenance costs. Especially since we do not see sweeping changes when testing this enterprise application.
Thank you in advance for your comments / advice!

You seem to have developed your test suites already. Now you are concerned about the maintenance in long term. Then I would advise you not to use the Spy and Recorder any longer.

These tools help you getting started quickly, but are not designed with “maintenance” in mind. It’s time for you to invest some days or weeks to study how to create test cases and test objects manually without the Spy and Recorder tools. You want to write test case scripts by text editor (in Script mode), want to create test objects manually one by one while you write locators in XPath or CSS selector. Do you need some tutorial for XPath? OK, this is it. You will name every single codes with identifiable names. You will create sub-folders to better organize the codes. Coding manually will motivate you to minimise the number of objects to develop. Programming is not so difficult. You may even find it a fun!

In the end, you would obtain far smaller number of test cases and test objects than what you have now — 233 test cases plus how many test objects do you have? 1000? Needless to say, it is easier to mainain smaller number of objects.

1 Like

@parsonschandler Great question.

I’ll agree 100% with @kazurayam and bolster it with this:

If you need to spend time maintaining tests you need to…

  1. Develop a robust and clear wait strategy. If you can’t explain your wait strategy in a sentence or two, you likely have a hodge-podge of guesses, fixed wait periods and wishful thinking. Spend most of your time refining your wait strategies, it will pay dividends later.
    In testing, WWW stands for World Wide Wait :nerd_face:

  2. Produce Tests Cases, Test Objects and all primary interfaces with clear and meaningful names. If you ever find yourself saying “I think it’s this one…” then your naming strategy is :poop: Choose a good name for “new thing” right up front. If you change those names a lot, you will lose test history. If historic statistics matter to you, that’s a huge loss.

  3. Understand that Testing is an Engineering Science. A toy like spy is not a scientific piece of engineering. I would not use it to get started let alone try to maintain its output.

  4. Understand that Bad Selectors are brittle selectors. Learn to code robust selectors that defend against document redesign and reshaping. Self-healing? I’ve never used it or needed it.

Good luck!

@kazurayam
Thank you for the response!
I should have mentioned this in my original message, but I have developed about 15 test cases so far and not the full 233. I’m at the point now where I’m going to dedicate alot of time over the coming weeks/months to get all the test cases built and wanted to know if I could be saving considerable amounts of time, but based on your comments, it looks like the work up front to apply proper naming is worthwhile. I tend to agree that organization on the front end can prevent a number of issues down the road. As for the comments on XPATH, up until this point, I have relied heavily on the Object Spy to help reduce the amount of time it takes to create the objects. I also like that it pulls a number of parent/child attributes for self healing. I don’t anticipate much of the HTML objects changing much since this is a mature enterprise application, but I would rather bet on the safe side! I have watched a number of videos on XPATH and find it to be a fairly straightforward process in identifying the right path, I would just rather rely on the tooling to help out to save some time until I become more proficient at locating them via the HTML manually.

I’m honestly shocked to hear that both of you don’t support the use of spy object and smart healing. I do understand that learning to find the objects manually is probably the more robust solution, but are these two tools within Katalon really that bad that they should be ignored completely? Or are you just saying that an experienced HTML automation engineer will always find the object better than AI?

This.

Don’t read any more into what I said than what I said.

I’m a chef. You want to eat ready-made meals. That’s fine.

1 Like

@parsonschandler

Please have a look at my previous posts.

@parsonschandler

You asked me if you should use Spy&Recorder tools or not.

It depends how the HTML of your Application Under Test is coded — is it coded in a way friendly to Spy&Recorder tools, or not?

I will present you a sample HTML that is friendly to the tools.

This URL https://katalon-demo-cura.herokuapp.com/profile.php#login is designed by Katalon team to demonstrate how Spy and Recorder tools work brilliant. Therefore they intentionally coded the HTML friendly to the tools.

Please read the HTML source code of this page. The HTML elements in this page are friendly to Spy, as the <input> and <button> elements have id attributes with fixed string values, as follows:

<input .. id="txt-username" ...>

Spy & Recorder will generate a XPath for this like:

//input[@id="txt-username"]

No doubt, this XPath expression is the best shortest, easiest to understand and most maintainable one. I, as an experienced XPath writer, will also suggest this expression.

However, the real world web applications are NOT designed as easy as this demo URL. The enterprise web app driven by sophisticated JavaScript libraries (React, Angular, etc) tend to generate HTML codes very difficult for Spy & Recorder tools to cope with.

First, often HTML elements lack ID attribute. In case a HTML element with an ID attribute, the value tend to be dynamic; the ID value changes every time you enquire the page. Please make a search in this Katalon Forum with a key “dynamic id”. You can find quite a lot of questions where people say the Spy & Recorder are not enough for their own cases.

Second, the Spy & Recorder tools are helpless for the HTML with <iframe> elements. If your AUT has any <iframe> tag in the HTML source, you need to do WebUI.switchToFrame() before selecting the element of your interest. Otherwise, your code will fail to find the element.

People who rely on the tools pay little attention to the HTML source. They don’t read the HTML of web pages. Therefore they are unaware if any <iframe> tags are there. And they raise questions to this forum saying “my test object fails to find the element, why?” again and again. All I can say to them is: “you should read your HTML carefully, otherwise you would never accomplish your job. The Spy & Recorder are not as intelligent as you innocently expect”.

How is the HTML of your Application Under Test is coded?

If your AUT is as friendly as https://katalon-demo-cura.herokuapp.com/profile.php#login, then Spy and Recorder will be good for you.

If your AUT is not designed with “E2E Testing using Tools” in mind, you need to be skilled for the XPath and/or CSS Selector technology anyway to accomplish your job.

I do not know how your HTML is coded. Therefore I have no definite advice to you which way to take.

Good luck. :wink:

I am not saying that these tools are bad. Just I am afraid that generative tools tend to put users off learning the valuable skills.

Every college student dreams of graduating summa cum laude (with highest praise). Others are pleased to graduate magna cum laude (with great praise). Still others are satisfied to graduate cum laude (with praise). Some are happy just to graduate at all!
WILLIAM E. LINNEY

How would I evaluate the XPath expressions generated by Spy & Recorder tools? – graduate sine laude (without praise). I wouldn’t rate it cum laude. If you study some text or book, you would be able to aim for magna cum laude or even higher. But once you are settled down on the tools, you would be no longer motivated for studying the arts. Of course, up to you. It depends on your AUT.

AI?

I have noticed some vendors (this and this) claim that word. But I have never seen any advertisement made by Katalon saying “Spy & Recorder are backed by AI”.

Where did you see it?

If there is any, I believe that it is a Vaporware, or a lie. The long list of questions at https://forum.katalon.com/search?q=dynamic%20id is enough to prove it.