[Sharing is learning] [Get rewards] Join Us Now to Become Katalon Champions. Learn more
huyuan
Katalon Apprentice
05/28/2018

Why Lose some suite report when execute suite collection in console mode?

Hello,
I execute a suite collection in console mode,which include 2 test suites,the cmd is as follow:
katalon -runMode=console -projectPath="F:\demo\demo.prj" -reportFolder="D:\project\Reports"  -retry=0 -testSuiteCollectionPath="Test Suites\C1" -browserType="Chrome" 

Some times, there is only one report folder generated in  D:\project\Reports,
while there should be 2 report folders, one report for every suite.
 And the log is like this :
--------------------------------------------------------------------------------
Test Suites/C1 - 20180528_160109....................2/2(100%)
--------------------------------------------------------------------------------
Test Suites/TS2_data - Chrome - 20180528_160110........................1/1(100%)
Test Suites/TS1 - Chrome - 20180528_160110.............................1/1(100%)
--------------------------------------------------------------------------------
And the execution mode is sequential.

Why  one report is lost? Is it because one test suite  report cover another suite report in the same collection? How can I solve the problem?

Upvote
Quote

Comments

  • David Edmondson
    Katalon Ambassador
    05/31/2018
    Try adding another argument to your command script (-reportName="Reports") and see if that helps...

    katalon -runMode=console -projectPath="F:\demo\demo.prj" -reportFolder="D:\project\Reports"  --reportName="Reports" retry=0 -testSuiteCollectionPath="Test Suites\C1" -browserType="Chrome" 
    Upvote
    Quote
  • Nick
    Katalon Apprentice
    06/01/2018
    I have also reported a similar problem in the forum (no response) with the docker image, which also runs in console mode. All suites write to the same report folder, so what you end up with is the results of the last test suite run. I believe it is a bug (although not confirmed), either way, it makes collections unusable.
    Upvote
    Quote
  • Gerard van der Winkel
    Katalon Ambassador
    06/02/2018
    The BUG here is that the timestamp in the report names are the same for different TestSuites in the TestSuiteCollection.

    The original reports are stil  generated correctly for Katalon, because they are placed SEPARATELY under subfolders (using TestSuite name) under the Katalon Project path.

    But when running under console mode these reports are copied ALL to the SAME supplied -reportFolder WITHOUT using the original subfolders name of the TestSuite, resulting in overwritting reports due to same timestamp name.

    Example: 

    >>>>>>
    C:\Katalon_Studio>katalon -noSplash  -runMode=console -projectPath="C:\Katalon_Studio_Projects\ESB API\ESB API.prj" -reportFolder="C:\Jenkins\workspace\Katalon Studio - ESB API\Reports" -reportFileName="report" -retry=0 -testSuiteCollectionPath="Test Suites/TSC-Regression" 
    Request sent successfully.
    Starting Groovy-Eclipse compiler resolver.  Specified compiler level: unspecified
    133 2.4.7.xx-201611170128-e46 = ACTIVE
    
    --------------------------------------------------------------------------------
    Test Suites/TSC-Regression - 20180601_105317.............................0/7(0%)
    --------------------------------------------------------------------------------
    Test Suites/TS-Regression/TS-Retrieve_PUK_Code - Chrome (headless) - 20180601_10
    5317.....................................................................0/2(0%)
    Test Suites/TS-Regression/TS-Retrieve_Balance_Info - Chrome (headless) - 2018060
    1_105317.................................................................0/2(0%)
    Test Suites/TS-Regression/TS-Query_MSISDN - Chrome (headless) - 20180601_105318.
    .........................................................................0/3(0%)
    --------------------------------------------------------------------------------
    
    Request sent successfully.
    
    --------------------------------------------------------------------------------
    Test Suites/TSC-Regression - 20180601_105317.............................0/7(0%)
    --------------------------------------------------------------------------------
    Test Suites/TS-Regression/TS-Retrieve_PUK_Code - Chrome (headless) - 20180601_10
    5317.....................................................................0/2(0%)
    Test Suites/TS-Regression/TS-Retrieve_Balance_Info - Chrome (headless) - 2018060
    1_105317.................................................................0/2(0%)
    Test Suites/TS-Regression/TS-Query_MSISDN - Chrome (headless) - 20180601_105318.
    .........................................................................0/3(0%)
    --------------------------------------------------------------------------------
    
    Copying report to folder C:\Jenkins\workspace\Katalon Studio - ESB API\Reports\20180601_105317...
    Uploading report to JIRA...
    Start sending test result to JIRA issues
    Report has been sent to JIRA
    
    --------------------------------------------------------------------------------
    Test Suites/TSC-Regression - 20180601_105317............................2/7(29%)
    --------------------------------------------------------------------------------
    Test Suites/TS-Regression/TS-Retrieve_Balance_Info - Chrome (headless) - 2018060
    1_105317...............................................................2/2(100%)
    Test Suites/TS-Regression/TS-Retrieve_PUK_Code - Chrome (headless) - 20180601_10
    5317.....................................................................0/2(0%)
    Test Suites/TS-Regression/TS-Query_MSISDN - Chrome (headless) - 20180601_105318.
    .........................................................................0/3(0%)
    --------------------------------------------------------------------------------
    
    Copying report to folder C:\Jenkins\workspace\Katalon Studio - ESB API\Reports\20180601_105317...
    Uploading report to JIRA...
    Start sending test result to JIRA issues
    Report has been sent to JIRA
    
    --------------------------------------------------------------------------------
    Test Suites/TSC-Regression - 20180601_105317............................4/7(57%)
    --------------------------------------------------------------------------------
    Test Suites/TS-Regression/TS-Retrieve_Balance_Info - Chrome (headless) - 2018060
    1_105317...............................................................2/2(100%)
    Test Suites/TS-Regression/TS-Retrieve_PUK_Code - Chrome (headless) - 20180601_10
    5317...................................................................2/2(100%)
    Test Suites/TS-Regression/TS-Query_MSISDN - Chrome (headless) - 20180601_105318.
    .........................................................................0/3(0%)
    --------------------------------------------------------------------------------
    
    Copying report to folder C:\Jenkins\workspace\Katalon Studio - ESB API\Reports\20180601_105318...
    Uploading report to JIRA...
    Start sending test result to JIRA issues
    Report has been sent to JIRA
    
    --------------------------------------------------------------------------------
    Test Suites/TSC-Regression - 20180601_105317...........................7/7(100%)
    --------------------------------------------------------------------------------
    Test Suites/TS-Regression/TS-Retrieve_Balance_Info - Chrome (headless) - 2018060
    1_105317...............................................................2/2(100%)
    Test Suites/TS-Regression/TS-Retrieve_PUK_Code - Chrome (headless) - 20180601_10
    5317...................................................................2/2(100%)
    Test Suites/TS-Regression/TS-Query_MSISDN - Chrome (headless) - 20180601_105318.
    .......................................................................3/3(100%)
    --------------------------------------------------------------------------------
    
    
    C:\Katalon_Studio>exit 1 
    Build step 'Execute Windows batch command' marked build as failure
    Recording test results
    Finished: FAILURE
    
    >>>>>>
    
    The first 2 TestSuites under this TestSuiteCollection BOTH get the same timestamp 20180601_105317 in their name !
    I would suggest that during the copying of the reports while running in console mode the subfolder for each TestSuite is also included.
    Upvote
    Quote
  • Vinh Nguyen
    Katalon Moderator
    06/25/2018
    This issue has been fixed and it will be delivered in the next release.
    Upvote
    Quote
  • Gerard van der Winkel
    Katalon Ambassador
    06/25/2018
    Looking forward to it. Thanx
    Upvote
    Quote
  • Swaminathan Nandakumar
    Katalon Apprentice
    07/11/2018
    Hello, I am facing the same problem but in the docker image (https://github.com/katalon-studio/docker-images).  Will you be updating that also as part of the release?

    Best, Nanda
    Upvote
    Quote
Sign In or Register to comment.