Jenkins cant find Junit reports


#1

Can anybody tell me the correct path to give Jenkins on windows to find the Junit_report.xml file generated after a test suite is executed?

I can see the file is here: C:\Users\Anthony\Katalon Studio\Examples\Reports\test\20180611_193541\JUnit_Report.xml

But pointing Jenkins at Reports\test\*\JUnit_Report.xml still causes a failure, I can skip the failure by telling Jenkins to ignore it but I want the reporting in Jenkins

Jenkins just tells me ‘Reports\test\*\JUnit_Report.xml’ doesn’t match anything: even ‘Reports’ doesn’t exist

The full path tells me C: doesnt exist, what am I doing wrong here please?

Prior to running the Katalon command i run cd C:\Katalon in the execute Windows batch command part. The Katalon test is running correctly and passing.

jenkinserror.png


#2

Figured it out, it was a case of adding this on to the windows batch command to run the test suite

-reportFolder="C:/Program Files (x86)/Jenkins/workspace/Katlon/Reports"

Katlon (typo I know) being my jenkins workspace folder and I created the Reports folder manually.

Then just setting the post build actions to look at

*/JUnit_Report.xml

And it works B)


#3

The above doesn’t work when using test collections with multiple suites as the reports go in the dynamically generated folders

So back to the original question, how do I point Jenkins to the correct folders for the xml reports

I thought Reports/*/Junit_Report.xml would do it but apparently not :’(


#4
  1. Jenkins’ Publish JUnit test result report uses as basedir of the fileset: the workspace root.
  2. Katalon’s example using simply Reports as foldername however uses Katalon project’s root folder.
  3. Due to a bug when running a Test Collection where reports are getting overwritten due to having the same filename, (Works fine for generating the Katalon’s files because they are placed under different subfolders, but when running Katalon in console mode with the Report options results in the end that the reports are copied all under one folder however),

Im now using the following way in Jenkins as a workaround.

I) Under Build Execute Windows batch command I’m using:

C:
cd C:\Katalon_Studio
katalon -noSplash  -runMode=console -projectPath="C:\Katalon_Studio_Projects\ESB API\ESB API.prj" -retry=0 -testSuiteCollectionPath="Test Suites/TSC-Regression"
SET LEVEL = %ERRORLEVEL%
robocopy "C:\Katalon_Studio_Projects\ESB API\Reports" "C:\Jenkins\workspace\Katalon Studio - ESB API\Reports" /E /COPYALL /V /NP /R:0
EXIT %LEVEL%

ofcourse this is just an example replace values as needed

Im not using the Report output options on the Katalon console mode command but just let all output be generated under their standard folders only and then uses robocopy to copy all new files (or just the needed JUnit xml files if you prefer) to the Jenkins project’s workspace

II) Under the Post-build Actions - Publish JUnit test result report:

I used following for “Test report XMLs” : Reports\\\*\JUnit_Report.xml

I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:

  1. There is NO subdivision for Test Suite names (TestCase name is re-used instead)
  2. Timings are not in milliseconds and Timings are missing on several levels.

#5

Gerard van der Winkel said:

  1. Jenkins’ Publish JUnit test result report uses as basedir of the fileset: the workspace root.
  2. Katalon’s example using simply Reports as foldername however uses Katalon project’s root folder.
  3. Due to a bug when running a Test Collection where reports are getting overwritten due to having the same filename, (Works fine for generating the Katalon’s files because they are placed under different subfolders, but when running Katalon in console mode with the Report options results in the end that the reports are copied all under one folder however),

Im now using the following way in Jenkins as a workaround.

I) Under Build Execute Windows batch command I’m using:

C:

cd C:\Katalon_Studio
katalon -noSplash -runMode=console -projectPath=“C:\Katalon_Studio_Projects\ESB API\ESB API.prj” -retry=0 -testSuiteCollectionPath=“Test Suites/TSC-Regression”
SET LEVEL = %ERRORLEVEL%
robocopy “C:\Katalon_Studio_Projects\ESB API\Reports” “C:\Jenkins\workspace\Katalon Studio - ESB API\Reports” /E /COPYALL /V /NP /R:0
EXIT %LEVEL%


  
ofcourse this is just an example replace values as needed  
  
Im **not** using the Report output options on the Katalon console mode command **but** just let all output be generated under their standard folders only and then uses robocopy to copy all new files  (or just the needed JUnit xml files if you prefer) to the Jenkins project's workspace  
  
II) Under the Post-build Actions - Publish JUnit test result report:  
  
I used following for "Test report XMLs" : Reports\\*\\*\\*\\JUnit_Report.xml  
  
I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:  
  
1) There is NO subdivision for Test Suite names (TestCase name is re-used instead)  
2) Timings are not in milliseconds and Timings are missing on several levels.

  

This is a great answer, thank you for the example, I will give it a try


#6

Hi Guys,

I am struck up with the same issue and all the steps mentioned here were useful.

But in my case the cp command is not getting executed after the Katalon command though the execution of tests is completed. the Jenkins is just running for ever without moving to next step.

Any idea why this is behaving like this?

Thanks,
Sampath


#7

Note: the Katalon command should NOT include the -noExit and -consoleLog parameters so that CI logs can be displayed directly on the job view.

see https://www.katalon.com/resources-center/tutorials/jenskins-ci-integration/ for more info.


#8

upvoted!


#9

Gerard van der Winkel said:

I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:

  1. There is NO subdivision for Test Suite names (TestCase name is re-used instead)
  2. Timings are not in milliseconds and Timings are missing on several levels.

Did you report #1 yet? I’m running into the same thing. I’m using a test suite collection to run the same test suite multiple times with different browsers and profiles, but each instance has the same exact name based on the test case.


#10

Greg Seaton said:

Did you report #1 yet? I’m running into the same thing.

Nope I did not had the time yet (other more important things to do in the meanwhile), but I will report it this weekend (I want to include some usefull info).