Katalon test results and Azure DevOps test run linking - I made a console app!

Product Forums Katalon Studio
, , ,
1

I wrote a small console app that will create Azure DevOps test runs and upload the results of your Katalon Test Suite results based on the report file contents.

There are a lot of areas where this app is not ideal but I’d like to think it’s a good starting point.

If anyone might find this helpful, feel free to try it out and let me know if you have any feedback.
Thanks!

  • Jeanie
5 Likes
2

Very nice, @jeanie.conner

I’m wondering if you thought of integrating the console app with the end of a suite run?

3

Thanks @Russ_Thomas

If I could have it run at the end of a suite that would be best as then I could fix the problem of potentially re-uploading old results, but I’m not sure how to do that without having to build a katalon plugin… in the future I can build a plugin, but since it requires Java I’ll have to rewrite the code as I wouldn’t be able to use the libraries for azure devops. It’s doable, though.

4

Keeping this at 10,000 ft…

  @AfterTestSuite
  def afterTestSuite(TestSuiteContext testSuiteContext) {
    runBatchFile("rowr111.bat")
  }

where runBatchFile is something like…

  /**
   * Execute a batch file situated in the KS project directory.
   * @param batchFile (String) e.g. "myfile.bat"
   */
  static void runBatchFile(String batchFile) {
    String bf = RunConfiguration.getProjectDir() + '/' + batchFile
    comment("Running batch file: " + bf)
    Runtime.runtime.exec(bf)
  }
5

for some reason I never noticed the Test Listener functionality before… :o I will try using that, thanks!

6

there is a small problem with this approach.
the app wrote by @jeanie.conner is parsing the JUnit_Report.xml file … which I don’t think it is available yet at the ‘AfterTestSuite’ stage. I have the feeling it is created after the testsuite scope is totally completed (not sure on this, I have to check)

So, a different approach may be needed. I think the data about executed testcases it is available in the testsuite context at this point … (again not sure,need to explore)

Another approach may be to use @AfterTestCase listener and update the results on the fly (status can be retrieved from testcase context) but for this the app code has to be re-implemented in katalon, e.g in a certain keyword (there is no need for a plugin)
In the end are just API calls so shouldn’t be very difficult.

7

@jeanie.conner

duh - I think you’re right. I seem to recall that biting me a few years ago – and one of the (many) reasons I decided to write my own reports.

Sorry Jeanie, didn’t mean to lead you on a fool’s errand.

There’s probably something you can do with a file watcher (maybe via groovyshell?) but not something I’ve tinkered with in groovy/java or even C#. If @anon46315158 is correct (pretty sure he is) , a file watcher would be the way to go.

8

thanks for all the info @Russ_Thomas and @anon46315158 - I’ll still look into how I can integrate it via keyword and the test listener without relying on the reports being generated, as that might be the most seamless way to do it. I think I’ll have to rewrite to call the rest API instead of using the client libraries to integrate with azure devops, but we’ll see.
Thanks again, will update later!

  • Jeanie