Allow optional custom JS function to calculate correct target CssPath

I have tried a couple of Web based macro recorders and it occurred to me that trying to guess the correct XPath or Css Target for page elements is hard, especially when the page was not built to account for automated testing or when the page is dynamic and it uses ID and Name attributes for its own purposes.

Manually correcting incorrect paths is also very frustrating and time consuming. Doing so takes all the fun out of using a macro recorder and it make the process very error prone.

My suggestion is to allow us to specify a custom in-page JavaScript function (by name) that would do the work of calculating the correct CSS Target based on what we know.

The benefit is that the page developer should be able to create such a function and if they can’t then they are properly incentivized to update their page structure to make things easier.

The macro recorder could then pass the page element to this function and the function would be responsible for returning a unique CSS Path.

  • If the function returned a NULL, then the recorder could use its default options.
  • If the function returned an Error then the recorder could show the error.
  • If the function returned a Path then the recorder could quickly verify that it is valid and that it returns the correct element.

Once a correct function was included on the page, then it should make the process of recording scripts much less painful and even fun.

Thank you very much.

Below I have included an example of such a function. The application I am testing purposely uses ID and Name attributes for its own purposes.

Note: I have included this function as an example only. Other web page structures would necessitate different logic.

// Generate a unique and short Css Path for an element.
// Does not use the ID or Name attributes.
function getCssPath(inputEl) {
    var curEl = inputEl;
    var elStack = [];
    var lastElsCount = null;  // Used to drop intermediate els that don't add to the specifity...
    var firstClass = function (el) {
        // Purposely only pick on the first class.
        return (el.classList && el.classList.length) ? el.classList[0] : "";
    };
    var elementsFound = function (cssPath) {
        try {
            var elsFound = document.querySelectorAll(cssPath);
            if (elsFound == null || elsFound.length < 1)
                throw "No elements found";
            return elsFound;
        } catch (err) {
            throw "elementsFound: Invalid Path [" + cssPath + "], " + ((typeof err == "string") ? err : err.message);
        }
    };
    var collapsePath = function (cssPath) {
        var pathSplit = cssPath.split('>');
        var newPath = '';
        var joinChar = '';
        for (var i = 0; i < pathSplit.length; i++) {
            if (pathSplit[i] == '')
                joinChar = ' ';
            else {
                newPath += joinChar + pathSplit[i];
                var joinChar = '>';
            }
        }
        return newPath;
    };
    while (curEl.parentNode != null) {
        var sibCount = 0;
        var sibIndex = 0;
        var curClass = firstClass(curEl);

        var curSelector = curEl.nodeName.toLowerCase() + ((curClass) ? '.' + curClass : "");

        // Try this combination.
        var combinedSelector = collapsePath((elStack.length) ? curSelector + ">" + elStack.join('>') : curSelector);
        var elsFound = elementsFound(combinedSelector);
        if (elsFound && elsFound.length == 1 && elsFound[0] === inputEl)
            return combinedSelector;

        // Add the nth-child to see if it is helpful.
        for (var i = 0; i < curEl.parentNode.childNodes.length; i++) {
            var sib = curEl.parentNode.childNodes[i];
            if (sib.nodeName == curEl.nodeName) {
                if (sib === curEl) {
                    sibIndex = sibCount;
                }
                sibCount++;
            }
        }
        if (sibCount > 1) {
            // Note: The 'nth-of-type' is a 1 based array
            curSelector += ':nth-of-type(' + (sibIndex + 1) + ')';

            // Try this combination.
            var combinedSelector = collapsePath((elStack.length) ? curSelector + ">" + elStack.join('>') : curSelector);
            var elsFound = elementsFound(combinedSelector);
            if (elsFound && elsFound.length == 1 && elsFound[0] === inputEl)
                return combinedSelector;
        }
        
        if (lastElsCount == null || lastElsCount > elsFound.length) {
            // If true then we added to the specificity
            lastElsCount = elsFound.length;
        }
        else {
            // The current selector did not help so add a null.
            curSelector = null; 
        }
        elStack.unshift(curSelector);
        curEl = curEl.parentNode;
    }

    // return stack.slice(1); // removes the html element
    var combinedSelector = collapsePath(elStack.slice(1).join('>'));
    if (!combinedSelector) {
        console.error(Error("Can't find Element Path"));
        debugger;
    }
    return combinedSelector;
}

Note: I have posted this same suggestion to the Kantu forum.

1 Like

Sorry, but I don’t understand. Your opening paragraph suggests a test engineer takes guesses at CSS selectors and XPath expressions – sorry, but I don’t believe that. And if it were true, then yes, I would agree, that’s hard – not least because it’s not how things are done.

Giving you the benefit of the doubt, I continued reading…

But you don’t say how this mechanism takes place. Is it not already possible to execute JavaScript using existing APIs?

I found that sentence difficult to parse…

Who benefits? The test engineer? The page developer?
Why do you think the page developer should do this? How are they incentivized? And to make what easier? If you are suggesting the developers will modify their structure to not use dynamic IDs (to make testing easier), what purpose does your function serve then?

Wait, you just said, “if they can’t then they are properly incentivized to update their page structure to make things easier.” In which case, why would “the macro recorder could then pass the page element to this function”? They made things easier, but still you want this function called somehow? Why?

Maybe it’s just me, but this proposal and its “logic” is really difficult to follow.

But anyway, continuing on…

CSS paths are not necessarily (and do not need to be and frequently aren’t) unique. How then should it be responsible for returning something that isn’t necessary?

Sorry. I’m completely lost.

Perhaps you might setup a use-case somewhere public and use this code to show your solution alongside a worked/annotated example. Perhaps then it might make some sense (to me, at least).

1 Like

Hello Russ and thanks for trying to read your way through.

When I said “takes guesses at CSS selectors and XPath expression”… I was referring to the macro recorder. Since the macro recorder has no special knowledge about the application it has to make it decisions based on what it can see + any hoped for best practices.

I then explain that manual correction is required when the macro recorder makes invalid assumptions about what to use in order to find the target element.

I am suggesting that the macro recorder be updated so that it directly asks the application during record time for the correct CSS Path. This could be implemented by having the macro recorder call a specified Javascript function on the page. That function would be written either by the tester or the page developer. The macro recorder would then pass the target element to the Javascript function and the page would return a CssPath that is stable, unique and appropriate for that application.

To quote myself…

This would replace the macro recorder attempting to use its own logic to build an appropriate XPath or CSS Path.

You indicate that CSS paths are not necessarily unique. That is true but it is very possible to create a CSS path that is unique and stable. Many of the test automation applications I have seen thus far allow them as a valid locator option.

If the page developer and tester CAN NOT create such a function then it is an obvious indication that the application needs work to make it testable. This puts the burden on the page developer instead of Katalon.

Do let me know if you have further questions.

1 Like

Now that, Anthony, is MUCH clearer. Thank you.

Disclaimer: I’m not a fan of the recorder.

I have no further questions but I do have an observation: I find anything that modifies the AUT (in your case, injection of JS to service the testing tool) dubious at best. True, I do that myself, a lot. But it’s still “bad”. If your extension were to suggest a way to load that mechanism from some external source then that would mitigate the issue in one step. (If it were possible, of course – which I would think it is)

Off topic: Have you tried cypress.io? Anyone?

I took a peek at Cypress.IO, and the docs indicate it already has this specific functionality I just suggested. Look for onElement.

Now I need to find out if it works as intended.

I haven’t tried cypress yet. I watched a few youtube videos many months ago and was somewhat impressed (enough that I haven’t forgotten it!)

I mentioned it to @YoungNgo in the hope that Katalon Studio could provide a way to host it allowing us to develop in Cypress where applicable. Haven’t heard anything back yet though.

If you try it, drop me a line and let me know how it went.

Cypress.io and Katalon are very different. I don’t imagine that they are easily compatible.

I was impressed by the Katalon Recorder but it did not work for me due to the issue mentioned above.

Cypress does not have any Macro recorder that I could find.

If you were talking about Katalon Recorder (not Katalon Studio), this feature has been around for a while.

https://docs.katalon.com/katalon-recorder/docs/extension-scripts-aka-user-extensionsjs-for-custom-locator-builders-and-actions.html

I have got it working by putting all of my locator code in an extension script and it works quite nicely now.

Thank you very much!

One related question: I could not get the extension script to call a javascript function within my application page. Is there a solution for that? My reason for doing so is that the locator code will likely be somewhat application version specific.

Anthony