Katalon Studio v5.8 - It's time for the all-new API testing experience
Anthony
Katalon Ambassador
06/11/2018
edited June 11

Jenkins cant find Junit reports

Can anybody tell me the correct path to give Jenkins on windows to find the Junit_report.xml file generated after a test suite is executed?

I can see the file is here: C:\Users\Anthony\Katalon Studio\Examples\Reports\test\20180611_193541\JUnit_Report.xml

But pointing Jenkins at Reports\test\*\JUnit_Report.xml still causes a failure, I can skip the failure by telling Jenkins to ignore it but I want the reporting in Jenkins



Jenkins just tells me ‘Reports\test\*\JUnit_Report.xml’ doesn’t match anything: even ‘Reports’ doesn’t exist

The full path tells me C: doesnt exist, what am I doing wrong here please?

Prior to running the Katalon command i run cd C:\Katalon in the execute Windows batch command part. The Katalon test is running correctly and passing.
Upvote
Quote

Comments

  • Anthony
    Katalon Ambassador
    06/11/2018
    edited June 11
    Figured it out, it was a case of adding this on to the windows batch command to run the test suite
    -reportFolder="C:/Program Files (x86)/Jenkins/workspace/Katlon/Reports"
    Katlon (typo I know) being my jenkins workspace folder and I created the Reports folder manually.

    Then just setting the post build actions to look at
    */JUnit_Report.xml

    And it works  B)

    Upvote
    Quote
  • Anthony
    Katalon Ambassador
    06/12/2018
    The above doesn't work when using test collections with multiple suites as the reports go in the dynamically generated folders

    So back to the original question, how do I point Jenkins to the correct folders for the xml reports

    I thought Reports/*/Junit_Report.xml would do it but apparently not  :'(  
    Upvote
    Quote
  • Gerard van der Winkel
    Katalon Ambassador
    06/13/2018
    edited June 13
    1) Jenkins' Publish JUnit test result report uses as basedir of the fileset: the workspace root.
    2) Katalon's example using simply Reports as foldername however uses Katalon project's root folder.
    3) Due to a bug when running a Test Collection where reports are getting overwritten due to having the same filename, (Works fine for generating the Katalon's files because they are placed under different subfolders, but when running Katalon in console mode with the Report options results in the end that the reports are copied all under one folder however),
      
    Im now using the following way in Jenkins as a workaround.

    I) Under Build Execute Windows batch command I'm using:

    C:
    cd C:\Katalon_Studio
    katalon -noSplash  -runMode=console -projectPath="C:\Katalon_Studio_Projects\ESB API\ESB API.prj" -retry=0 -testSuiteCollectionPath="Test Suites/TSC-Regression"
    SET LEVEL = %ERRORLEVEL%
    robocopy "C:\Katalon_Studio_Projects\ESB API\Reports" "C:\Jenkins\workspace\Katalon Studio - ESB API\Reports" /E /COPYALL /V /NP /R:0
    EXIT %LEVEL%

    ofcourse this is just an example replace values as needed

    Im not using the Report output options on the Katalon console mode command but just let all output be generated under their standard folders only and then uses robocopy to copy all new files  (or just the needed JUnit xml files if you prefer) to the Jenkins project's workspace

    II) Under the Post-build Actions - Publish JUnit test result report:

    I used following for "Test report XMLs" : Reports\*\*\*\JUnit_Report.xml

    I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:

    1) There is NO subdivision for Test Suite names (TestCase name is re-used instead)
    2) Timings are not in milliseconds and Timings are missing on several levels.
    Upvote
    Quote
  • Anthony
    Katalon Ambassador
    06/13/2018
    1) Jenkins' Publish JUnit test result report uses as basedir of the fileset: the workspace root.
    2) Katalon's example using simply Reports as foldername however uses Katalon project's root folder.
    3) Due to a bug when running a Test Collection where reports are getting overwritten due to having the same filename, (Works fine for generating the Katalon's files because they are placed under different subfolders, but when running Katalon in console mode with the Report options results in the end that the reports are copied all under one folder however),
      
    Im now using the following way in Jenkins as a workaround.

    I) Under Build Execute Windows batch command I'm using:

    C:
    cd C:\Katalon_Studio
    katalon -noSplash  -runMode=console -projectPath="C:\Katalon_Studio_Projects\ESB API\ESB API.prj" -retry=0 -testSuiteCollectionPath="Test Suites/TSC-Regression"
    SET LEVEL = %ERRORLEVEL%
    robocopy "C:\Katalon_Studio_Projects\ESB API\Reports" "C:\Jenkins\workspace\Katalon Studio - ESB API\Reports" /E /COPYALL /V /NP /R:0
    EXIT %LEVEL%

    ofcourse this is just an example replace values as needed

    Im not using the Report output options on the Katalon console mode command but just let all output be generated under their standard folders only and then uses robocopy to copy all new files  (or just the needed JUnit xml files if you prefer) to the Jenkins project's workspace

    II) Under the Post-build Actions - Publish JUnit test result report:

    I used following for "Test report XMLs" : Reports\*\*\*\JUnit_Report.xml

    I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:

    1) There is NO subdivision for Test Suite names (TestCase name is re-used instead)
    2) Timings are not in milliseconds and Timings are missing on several levels.
    This is a great answer, thank you for the example, I will give it a try
    Upvote
    Quote
  • Sampath Kumar
    Katalon Apprentice
    08/17/2018
    Hi Guys,

    I am struck up with the same issue and all the steps mentioned here were useful.

    But in my case the cp command is not getting executed after the Katalon command though the execution of tests is completed. the Jenkins is just running for ever without moving to next step.

    Any idea why this is behaving like this?

    Thanks,
    Sampath
    Upvote
    Quote
  • Gerard van der Winkel
    Katalon Ambassador
    08/29/2018
    Note: the Katalon command should NOT include the -noExit and -consoleLog parameters so that CI logs can be displayed directly on the job view.

    see https://www.katalon.com/resources-center/tutorials/jenskins-ci-integration/ for more info.
    Upvote
    Quote
  • kazurayam
    Katalon Evangelist
    08/29/2018
    upvoted!
    Upvote
    Quote
  • Greg Seaton
    Katalon Apprentice
    10/17/2018
    I now only have one issue still though with the JUnit_Report.xmls that are generated (I need to make a BUG report for that today): Info in xml is inconvenient/incomplete:

    1) There is NO subdivision for Test Suite names (TestCase name is re-used instead)
    2) Timings are not in milliseconds and Timings are missing on several levels.
    Did you report #1 yet? I'm running into the same thing. I'm using a test suite collection to run the same test suite multiple times with different browsers and profiles, but each instance has the same exact name based on the test case.
    Upvote
    Quote
  • Gerard van der Winkel
    Katalon Ambassador
    10/18/2018
    Did you report #1 yet? I'm running into the same thing. 
    Nope I did not had the time yet (other more important things to do in the meanwhile), but I will report it this weekend (I want to include some usefull info).
    Upvote
    Quote
Sign In or Register to comment.