XML/JUnit reports no longer working? KS7 RE


#1

I have installed the Basic Report plugin, however, in Katalon, there is no option for XML or JUnit. It only shows HTML, CSV, and PDF. I want to be able to add a post-build action in Jenkins to generate a JUnit report, but the directions given on the Katalon plugin are no longer accurate (as JUnit is not showing as an option).

Win10, KS7 RE


#2

Hi. In KS and RE 7 JUnit reports are always generated and the plugin is not required for that.


#3

Thank you for the reply! I must be doing something wrong with my Jenkins configuration then because I can’t seem to get them to show up. Could you advise on what I should put for Test report XMLs on Jenkins? I had tried Reports/**/JUnit_Report.xml but I get

Reports/**/JUnit_Report.xml’ doesn’t match anything: ‘Reports’ exists but not ‘Reports/**/JUnit_Report.xml


#4

Here is the console output from a sample test suite that I’ve got to try to see how to make it work:

    00:00:00.002 Started by user Jess Palmer
    00:00:00.008 Building in workspace C:\Program Files (x86)\Jenkins\workspace\Test Project
    00:00:00.013 using credential 96693d5d-1c7f-473c-842f-76fd869aecf8
    00:00:00.027  > C:\Program Files\Git\cmd\git.exe rev-parse --is-inside-work-tree # timeout=10
    00:00:00.079 Fetching changes from the remote Git repository
    00:00:00.079  > C:\Program Files\Git\cmd\git.exe config remote.origin.url https://github.com/Client-ebiz/magento2_automated_testing.git # timeout=10
    00:00:00.135 Fetching upstream changes from https://github.com/Client-ebiz/magento2_automated_testing.git
    00:00:00.136  > C:\Program Files\Git\cmd\git.exe --version # timeout=10
    00:00:00.185 using GIT_ASKPASS to set credentials 
    00:00:00.188  > C:\Program Files\Git\cmd\git.exe fetch --tags --force --progress https://github.com/Client-ebiz/magento2_automated_testing.git +refs/heads/*:refs/remotes/origin/*
    00:00:01.419  > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
    00:00:01.469  > C:\Program Files\Git\cmd\git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10
    00:00:01.683 Checking out Revision f12904f9d0daa318a5322c1f25a05494bdb4ec00 (refs/remotes/origin/master)
    00:00:01.683  > C:\Program Files\Git\cmd\git.exe config core.sparsecheckout # timeout=10
    00:00:01.783  > C:\Program Files\Git\cmd\git.exe checkout -f f12904f9d0daa318a5322c1f25a05494bdb4ec00
    00:00:01.959 Commit message: "tiny ts"
    00:00:01.963  > C:\Program Files\Git\cmd\git.exe rev-list --no-walk f12904f9d0daa318a5322c1f25a05494bdb4ec00 # timeout=10
    00:00:02.146 [Test Project] $ cmd /c call C:\WINDOWS\TEMP\jenkins5523489595784282563.bat
    00:00:02.195 
    00:00:02.195 C:\Program Files (x86)\Jenkins\workspace\Test Project>C:
    00:00:02.195 
    00:00:02.195 C:\Program Files (x86)\Jenkins\workspace\Test Project>cd \Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3 
    00:00:02.196 
    00:00:02.196 C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3>katalonc -noSplash -runMode=console -projectPath="C:\Users\jpalmer\Katalon Studio\M2_Automation\M2_Automation.prj" -retry=0 -testSuitePath="Test Suites/tinyTestSuite" -executionProfile="default" -browserType="Chrome (headless)" -apiKey="304fa0ab-75a6-4300-9a02-d898366fc4aa" 
    00:00:05.271 Katalon workspace folder is set to default location: C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3\config
    00:00:15.436 Starting Groovy-Eclipse compiler resolver.  Specified compiler level: unspecified
    00:00:15.437 171 2.4.7.xx-201611170128-e46 = ACTIVE
    00:00:16.019 
    00:00:16.019 
    00:00:16.023 INFO: Katalon Version: 7.0.4
    00:00:16.027 INFO: Command-line arguments: -runMode=console -projectPath=C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Client_M2_Automation.prj -retry=0 -testSuitePath=Test Suites/tinyTestSuite -executionProfile=default -browserType=Chrome (headless) -apiKey=******
    00:00:16.030 INFO: User working dir: C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3
    00:00:16.042 INFO: TestOps server URL: https://analytics.katalon.com
    00:00:16.047 INFO: Katalon Store server URL: https://store.katalon.com
    00:00:16.051 INFO: User home: C:\WINDOWS\system32\config\systemprofile
    00:00:16.055 INFO: Java vendor: Oracle Corporation
    00:00:16.058 INFO: Java version: 1.8.0_181
    00:00:16.061 INFO: Local OS: Windows 10 64bit
    00:00:16.564 INFO: CPU load: 41%
    00:00:16.567 INFO: Total memory: 7888 MB
    00:00:16.571 INFO: Free memory: 1170 MB
    00:00:16.654 INFO: Machine ID: ab0a3d115c3a09a59cc22229eb4922c3
    00:00:16.663 
    00:00:16.664 
    00:00:16.674 Activating...
    00:00:16.686 Start activating offline...
    00:00:16.689 Finding valid offline licenses in folder: C:\WINDOWS\system32\config\systemprofile\.katalon\license
    00:00:16.695 The number of valid offline licenses: 0
    00:00:16.698 Offline activation failed.
    00:00:16.698 Start activating online...
    00:00:21.080 Delete folder: bin
    00:00:21.563 Delete folder: Libs
    00:00:21.590 Cleaning up workspace
    00:00:21.696 Opening project file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Client_M2_Automation.prj
    00:02:14.186 /C%%Users%jpalmer%Katalon Studio%Client_M2_Automation%Client_M2_Automation.prj/Keywords/client/clientUtils.groovy: 24 Groovy:unable to resolve class internal.GlobalVariable
    00:02:17.398 Generating global variables...
    00:02:18.576 /C%%Users%jpalmer%Katalon Studio%Client_M2_Automation%Client_M2_Automation.prj/Keywords/client/clientUtils.groovy: 24 Groovy:unable to resolve class internal.GlobalVariable
    00:02:22.293 Parsing custom keywords...
    00:02:27.917 Project 'Client_M2_Automation' opened
    00:02:27.950 Start reloading plugins...
    00:02:28.613 Katalon version: 7.0.4
    00:02:28.613 Plugin info URL: https://store.katalon.com/api/products/ks?appVersion=7.0.4&appType=ENGINE&licenseType=TRIAL
    00:02:28.630 Plugin info: {
    00:02:28.630   "expired": false,
    00:02:28.630   "productId": 59,
    00:02:28.630   "name": "Basic Report",
    00:02:28.630   "id": 32340
    00:02:28.630 }
    00:02:28.631 Plugin info: {
    00:02:28.631   "expired": false,
    00:02:28.631   "productId": 69,
    00:02:28.631   "name": "UploadFile Keywords",
    00:02:28.631   "id": 27646
    00:02:28.631 }
    00:02:28.632 Plugin info: {
    00:02:28.632   "expired": true,
    00:02:28.632   "productId": 3,
    00:02:28.632   "name": "Jira Integration",
    00:02:28.632   "id": 5771
    00:02:28.632 }
    00:02:28.648 Plugin name: Basic Report. Is online: true. Plugin location: C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3\configuration\plugin\install\32340\1.0.7\katalon-studio-report-plugin.jar.
    00:02:28.650 Plugin name: UploadFile Keywords. Is online: true. Plugin location: C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3\configuration\plugin\install\27646\1.0.0\katalon-studio-uploadfile-plugin.jar.
    00:02:28.650 Expired plugin: 5771.
    00:02:33.657 Plugin "Basic Report" has been installed.
    00:02:33.657 Plugin "UploadFile Keywords" has been installed.
    00:02:33.657 Plugin "Jira Integration" has been uninstalled.
    00:02:33.679 Start check license task
    00:02:34.723 chromedriver is located at default location: C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3\configuration\resources\drivers\chromedriver_win32\chromedriver.exe. In case your browser is updated to a newer version, please use this command to update chromdriver: --config -webui.autoUpdateDrivers=true
    00:02:38.139 
    00:02:38.139 --------------------------------------------------------------------------------
    00:02:38.139 Test Suites/tinyTestSuite - Chrome (headless) - 20191112_161933..........0/1(0%)
    00:02:38.139 --------------------------------------------------------------------------------
    00:02:38.139 
    00:02:42.396 2019-11-12 16:19:40.881 INFO  c.k.katalon.core.main.TestSuiteExecutor  - START Test Suites/tinyTestSuite
    00:02:42.469 2019-11-12 16:19:40.962 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostName = T450S-W10-JPALM$ - T450S-W10-JPalmer.lokion.com
    00:02:42.473 2019-11-12 16:19:40.965 INFO  c.k.katalon.core.main.TestSuiteExecutor  - os = Windows 10 64bit
    00:02:42.474 2019-11-12 16:19:40.967 INFO  c.k.katalon.core.main.TestSuiteExecutor  - hostAddress = 10.80.1.75
    00:02:42.493 2019-11-12 16:19:40.976 INFO  c.k.katalon.core.main.TestSuiteExecutor  - katalonVersion = 7.0.4.1
    00:02:43.044 2019-11-12 16:19:41.537 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
    00:02:43.044 2019-11-12 16:19:41.538 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/client404Page
    00:02:43.877 2019-11-12 16:19:42.370 DEBUG testcase.client404Page                      - 1: openBrowser(baseurl, CONTINUE_ON_FAILURE)
    00:02:44.682 2019-11-12 16:19:43.176 INFO  c.k.k.core.webui.driver.DriverFactory    - Starting 'Chrome (headless)' driver
    00:02:45.157 Nov 12, 2019 4:19:43 PM org.openqa.selenium.remote.DesiredCapabilities chrome
    00:02:45.157 INFO: Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
    00:02:45.157 2019-11-12 16:19:43.609 INFO  c.k.k.core.webui.driver.DriverFactory    - Action delay is set to 0 seconds
    00:02:46.554 Starting ChromeDriver 77.0.3865.40 (f484704e052e0b556f8030b65b953dce96503217-refs/branch-heads/3865@{#442}) on port 24959
    00:02:46.554 Only local connections are allowed.
    00:02:46.555 Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
    00:02:48.122 [1573597186.615][WARNING]: This version of ChromeDriver has not been tested with Chrome version 78.
    00:02:50.127 [1573597188.621][WARNING]: Timed out connecting to Chrome, retrying...
    00:02:52.367 Nov 12, 2019 4:19:50 PM org.openqa.selenium.remote.ProtocolHandshake createSession
    00:02:52.367 INFO: Detected dialect: W3C
    00:02:52.516 2019-11-12 16:19:51.006 INFO  c.k.k.core.webui.driver.DriverFactory    - sessionId = e63bab129b125d565edebd8f6c9fe2d6
    00:02:52.577 2019-11-12 16:19:51.070 INFO  c.k.k.core.webui.driver.DriverFactory    - browser = Chrome 78.0.3904.97
    00:02:52.584 2019-11-12 16:19:51.071 INFO  c.k.k.core.webui.driver.DriverFactory    - platform = Windows 10
    00:02:52.584 2019-11-12 16:19:51.077 INFO  c.k.k.core.webui.driver.DriverFactory    - seleniumVersion = 3.141.59
    00:02:52.584 2019-11-12 16:19:51.078 INFO  c.k.k.core.webui.driver.DriverFactory    - proxyInformation = ProxyInformation{proxyOption=NO_PROXY, proxyServerType=HTTP, password=, proxyServerAddress=, proxyServerPort=0}
    00:02:53.139 
    00:02:53.139 --------------------------------------------------------------------------------
    00:02:53.139 Test Suites/tinyTestSuite - Chrome (headless) - 20191112_161933..........0/1(0%)
    00:02:53.139 --------------------------------------------------------------------------------
    00:02:53.139 
    00:02:57.123 2019-11-12 16:19:55.617 DEBUG testcase.client404Page                      - 2: setViewPortSize(1600, 1200, CONTINUE_ON_FAILURE)
    00:02:59.282 [1573597197.776][WARNING]: Timed out connecting to Chrome, retrying...
    00:03:01.695 2019-11-12 16:20:00.188 DEBUG testcase.client404Page                      - 3: navigateToUrl("https://staging.client.com/404", STOP_ON_FAILURE)
    00:03:05.165 2019-11-12 16:20:03.658 DEBUG testcase.client404Page                      - 4: client.clientUtils.checkHeader()
    00:03:07.950 2019-11-12 16:20:06.444 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - client.clientUtils.checkHeader is PASSED
    00:03:07.951 2019-11-12 16:20:06.444 DEBUG testcase.client404Page                      - 5: client.clientUtils.checkFooter()
    00:03:08.141 
    00:03:08.141 --------------------------------------------------------------------------------
    00:03:08.141 Test Suites/tinyTestSuite - Chrome (headless) - 20191112_161933..........0/1(0%)
    00:03:08.141 --------------------------------------------------------------------------------
    00:03:08.141 
    00:03:14.010 2019-11-12 16:20:12.503 INFO  c.k.k.c.webui.common.WebUiCommonHelper   - Unable to find the element located by 'By.xpath: (.//*[normalize-space(text()) and normalize-space(.)='Rebates Link Placeholder'])[1]/following::li[1]'. Please recheck the objects properties to make sure the desired element is located. 
    00:03:21.159 2019-11-12 16:20:19.653 INFO  c.k.k.c.webui.common.WebUiCommonHelper   - Unable to find the element located by 'By.xpath: (.//*[normalize-space(text()) and normalize-space(.)='Control your yard, anytime, anywhere'])[1]/following::h3[1]'. Please recheck the objects properties to make sure the desired element is located. 
    00:03:22.353 2019-11-12 16:20:20.847 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - client.clientUtils.checkFooter is PASSED
    00:03:22.354 2019-11-12 16:20:20.848 DEBUG testcase.client404Page                      - 6: verifyElementPresent(findTestObject("page404/errorImage"), 30, STOP_ON_FAILURE)
    00:03:22.519 2019-11-12 16:20:21.013 DEBUG testcase.client404Page                      - 7: verifyElementPresent(findTestObject("page404/header1"), 30, CONTINUE_ON_FAILURE)
    00:03:22.836 2019-11-12 16:20:21.330 DEBUG testcase.client404Page                      - 8: verifyElementText(findTestObject("page404/header2"), "The page you are looking for may have moved or no longer exists.", CONTINUE_ON_FAILURE)
    00:03:23.032 2019-11-12 16:20:21.525 DEBUG testcase.client404Page                      - 9: verifyElementText(findTestObject("page404/bulletA"), "If you typed the URL directly, please make sure the spelling is correct.", CONTINUE_ON_FAILURE)
    00:03:23.141 
    00:03:23.141 --------------------------------------------------------------------------------
    00:03:23.141 Test Suites/tinyTestSuite - Chrome (headless) - 20191112_161933..........0/1(0%)
    00:03:23.141 --------------------------------------------------------------------------------
    00:03:23.142 
    00:03:23.186 2019-11-12 16:20:21.679 DEBUG testcase.client404Page                      - 10: verifyElementText(findTestObject("page404/bulletB"), "If you are not sure how you got here, go back to the previous page or return to our client� homepage.", CONTINUE_ON_FAILURE)
    00:03:23.341 2019-11-12 16:20:21.835 DEBUG testcase.client404Page                      - 11: verifyElementPresent(findTestObject("page404/clientLink"), 30, CONTINUE_ON_FAILURE)
    00:03:23.463 2019-11-12 16:20:21.956 DEBUG testcase.client404Page                      - 12: closeBrowser(CONTINUE_ON_FAILURE)
    00:03:24.813 2019-11-12 16:20:23.307 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/client404Page
    00:03:25.054 2019-11-12 16:20:23.547 INFO  com.kms.katalon.core.util.KeywordUtil    - Start generating HTML report folder at: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933...
    00:03:25.085 2019-11-12 16:20:23.573 INFO  com.kms.katalon.core.util.KeywordUtil    - HTML report generated
    00:03:25.181 2019-11-12 16:20:23.675 INFO  c.k.katalon.core.main.TestSuiteExecutor  - --------------------
    00:03:25.182 2019-11-12 16:20:23.676 INFO  c.k.katalon.core.main.TestSuiteExecutor  - END Test Suites/tinyTestSuite
    00:03:25.182 2019-11-12 16:20:23.676 INFO  c.k.katalon.core.main.TestSuiteExecutor  - ====================
    00:03:25.213 Launcher status after execution process completed: Running
    00:03:26.096 Start writing execution.uuid file to folder: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933
    00:03:26.148 Start generating JUnit report folder at: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933...
    00:03:26.230 JUnit report were generated at folder: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933
    00:03:26.246 Uploading report to Katalon TestOps...
    00:03:26.249 Start sending test result to Katalon TestOps
    00:03:26.585 Uploading log files of test suite
    00:03:26.738 Sending file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933\execution.uuid
    00:03:27.291 Sending file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933\execution0.log
    00:03:27.490 Sending file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933\execution.properties
    00:03:27.662 Sending file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933\JUnit_Report.xml
    00:03:27.871 Sending file: C:\Users\jpalmer\Katalon Studio\Client_M2_Automation\Reports\20191112_161932\tinyTestSuite\20191112_161933\20191112_161933.html
    00:03:28.119 TestOps: Start uploading report to Katalon TestOps server: https://analytics.katalon.com
    00:03:28.617 Katalon TestOps - Execution URL: https://analytics.katalon.com/from-ks/execution-result?teamId=16965&projectId=23308&id=4544745
    00:03:28.617 End sending to Katalon TestOps
    00:03:28.617 Report has been sent to Katalon TestOps
    00:03:38.143 
    00:03:38.143 --------------------------------------------------------------------------------
    00:03:38.143 Test Suites/tinyTestSuite - Chrome (headless) - 20191112_161933........1/1(100%)
    00:03:38.143 --------------------------------------------------------------------------------
    00:03:38.143 
    00:03:38.144 Execution completed. Exit code: 0.
    00:03:38.146 Start clean up session
    00:03:38.148 End check license task
    00:03:38.153 End clean up session
    00:03:38.158 Start release license task
    00:03:39.179 License released
    00:03:39.185 End release license task
    00:03:39.194 All launchers terminated
    00:03:40.418 
    00:03:40.418 C:\Users\jpalmer\Desktop\Katalon_Studio_Windows_64-6.1.3\Katalon_Studio_Windows_64-6.1.3>exit 0 
    00:03:40.731 Recording test results
    00:03:40.735 ERROR: Step ‘Publish JUnit test result report’ failed: No test report files were found. Configuration error?
    00:03:40.742 Finished: FAILURE

#5

hello,

I am using RE 7.0.4
no issues with JUnit reports in Jenkins
this is how i’ll run jenkins job in windws

cd C:\Users\xxx\KatalonStudio\Katalon_Studio_Engine_Windows_64-7.0.4
katalonc -noSplash -runMode=console -reportFolder="%WORKSPACE%/Reports" -projectPath="%WORKSPACE%\KatalonProject.prj" -retry=0 -testSuiteCollectionPath=“Test Suites/RegressionTestSuiteCollection” -browserType=“Chrome (headless)” -apiKey=xxxxxx
if ERRORLEVEL 1 set ERRORLEVEL=0


#6

Adding this to my command worked, thank you!