Running two test cases simultaneously - Ping Pong Test Case

jai8.jacob · August 28, 2019, 11:40pm

Hi all
I want to test a messaging Application on Android. Let’s say its similar to WhatsApp. I would like to have a Ping Pong(this is made up) test scenario where the App is installed on two phones A and B. A sends a message to B, once a message is received B would reply until then B would just wait. This should happen for, let’s say 100 iterations.

A basic version of what I want to achieve would look somewhat like the image below [for reference]:

What do I know so far?
I am aware from a different post that it is possible to “generate command” Katalon and then one can create a batch file that would invoke the scripts mentioned in the command that was generated. I am not sure how to do it step by step. I seem to miss something cause my batch file produces no output when I execute the batch file.

So my plan was to create two of these batch files namely, ping.bat and pong.bat and execute them one after the other.

What do I need help with?
I’ve got the test script that generates the messages with the count. So I’ve got 2 test scripts namely, ping and pong. How do I run them simultaneously on my Test PC setup that runs Katalon 6.2.1.

Test Setup
1 PC(Windows 10, 8GB RAM, i7), 2 Smart phones(Google Pixel XL, Samsung S8), Katalon Studio 6.2.1, 2 Test Scripts.

Any leads on how can I make this happen would be much helpful. Thanks in advance.

jj

Ibus · August 29, 2019, 5:05am

make a global variabile, gave it value 0
from tescase 1, do the ping and call testcase 2, if the counter is less than 100, otherwise stop
from testcase 2, do the pong and call testcase1.

put testcase1 in a suite and run it.

without the counter … will run forever

Ibus · August 29, 2019, 9:32am

you can do it also without globals, using testcase variables. declare the count in variables tab on both testcases, and call each-other with variable binding,so you export the current counter value between each other. doing it like this you no longer need a test suite and is not dependant on the selected profile

jai8.jacob · September 3, 2019, 6:26am

Thanks for your time @Ibus . Appreciate it.

From what I understand, To achieve the Ping-Pong scenario I need to create two Test Cases and have a single variable named ‘count’ as a shared variable. I could possibly do that in Java if I were to write classes and an Interface. I am unsure how would I specifically achieve this in Katalon Studio? I am a bit confused. If you could show me a sample or point me to a page that can help me get this(“count” variable binding) done that would be great!

Ibus · September 4, 2019, 8:29am

well … here a proof of concept.
First, I made two testcases, ‘TC ping’ and ‘TC pong’. Both of them have a public variable, ‘count’, of type number with initial value 0 (use the Variables tab to define them). See attached pic:

ping_pong

Next, for fun, let’s make a custom ‘test’ package with two keywords, ping and pong, which only prints the received count value <<< you can put your scripts there:

package test

import com.kms.katalon.core.annotation.Keyword

public class pp {
	@Keyword
	def ping(count){
		println("ping! ${count}")
	}
	
	@Keyword
	def pong(count){
		println("pong! ${count}")
	}
}

Now, the tescases looks as follows:

in ping we check if we reached the iteration limit, increase the count, do the ping and call the pong testcase. we bind the count variable to save the actual value.

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

if (count < 5) {
	count ++
	CustomKeywords.'test.pp.ping'(count)
	WebUI.callTestCase(findTestCase('TC pong'), ["count":count])
} else {
	println('done!')
}

in pong, we have to make sure that the case is called by the first one, so we check the count to not be zero, do the pong and call back the ping testcase. we bind again the count variable to pass back the actual value.

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

if (count != 0) {
	CustomKeywords.'test.pp.pong'(count)
	WebUI.callTestCase(findTestCase('TC ping'), ["count":count])
} else {
	println('dont start the loop from this testcase!')
}

and we start the loop by running ‘TC ping’ (attempting to start the loop with 'TC pong" will just end quickly)
The Log View will be funny, since each case will appear under the previous one, as a tree. I wonder how will look for 100 iterations

Here is the console output:

2019-09-04 11:35:22.116 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:22.119 INFO  c.k.katalon.core.main.TestCaseExecutor   - START Test Cases/TC ping
2019-09-04 11:35:22.402 INFO  c.k.katalon.core.main.TestCaseExecutor   - (Default) count = 0
2019-09-04 11:35:22.593 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:22.595 DEBUG testcase.TC ping                         - 1: (count++)
2019-09-04 11:35:22.597 DEBUG testcase.TC ping                         - 2: test.pp.ping(count)
ping! 1
2019-09-04 11:35:22.626 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.ping is PASSED
2019-09-04 11:35:22.627 DEBUG testcase.TC ping                         - 3: callTestCase(findTestCase("TC pong"), ["count":count])
2019-09-04 11:35:22.767 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:22.767 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC pong
2019-09-04 11:35:22.773 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 1
2019-09-04 11:35:22.831 DEBUG testcase.TC pong                         - 1: if (count != 0)
2019-09-04 11:35:22.832 DEBUG testcase.TC pong                         - 1: test.pp.pong(count)
pong! 1
2019-09-04 11:35:22.833 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.pong is PASSED
2019-09-04 11:35:22.833 DEBUG testcase.TC pong                         - 2: callTestCase(findTestCase("TC ping"), ["count":count])
2019-09-04 11:35:22.899 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:22.899 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC ping
2019-09-04 11:35:22.904 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 1
2019-09-04 11:35:22.905 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:22.906 DEBUG testcase.TC ping                         - 1: (count++)
2019-09-04 11:35:22.907 DEBUG testcase.TC ping                         - 2: test.pp.ping(count)
ping! 2
2019-09-04 11:35:22.908 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.ping is PASSED
2019-09-04 11:35:22.909 DEBUG testcase.TC ping                         - 3: callTestCase(findTestCase("TC pong"), ["count":count])
2019-09-04 11:35:22.969 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:22.969 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC pong
2019-09-04 11:35:22.973 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 2
2019-09-04 11:35:22.976 DEBUG testcase.TC pong                         - 1: if (count != 0)
2019-09-04 11:35:22.976 DEBUG testcase.TC pong                         - 1: test.pp.pong(count)
pong! 2
2019-09-04 11:35:22.977 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.pong is PASSED
2019-09-04 11:35:22.978 DEBUG testcase.TC pong                         - 2: callTestCase(findTestCase("TC ping"), ["count":count])
2019-09-04 11:35:23.038 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.038 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC ping
2019-09-04 11:35:23.045 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 2
2019-09-04 11:35:23.046 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:23.047 DEBUG testcase.TC ping                         - 1: (count++)
2019-09-04 11:35:23.048 DEBUG testcase.TC ping                         - 2: test.pp.ping(count)
ping! 3
2019-09-04 11:35:23.049 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.ping is PASSED
2019-09-04 11:35:23.050 DEBUG testcase.TC ping                         - 3: callTestCase(findTestCase("TC pong"), ["count":count])
2019-09-04 11:35:23.110 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.110 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC pong
2019-09-04 11:35:23.114 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 3
2019-09-04 11:35:23.115 DEBUG testcase.TC pong                         - 1: if (count != 0)
2019-09-04 11:35:23.116 DEBUG testcase.TC pong                         - 1: test.pp.pong(count)
pong! 3
2019-09-04 11:35:23.116 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.pong is PASSED
2019-09-04 11:35:23.116 DEBUG testcase.TC pong                         - 2: callTestCase(findTestCase("TC ping"), ["count":count])
2019-09-04 11:35:23.172 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.172 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC ping
2019-09-04 11:35:23.175 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 3
2019-09-04 11:35:23.177 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:23.177 DEBUG testcase.TC ping                         - 1: (count++)
2019-09-04 11:35:23.178 DEBUG testcase.TC ping                         - 2: test.pp.ping(count)
ping! 4
2019-09-04 11:35:23.178 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.ping is PASSED
2019-09-04 11:35:23.179 DEBUG testcase.TC ping                         - 3: callTestCase(findTestCase("TC pong"), ["count":count])
2019-09-04 11:35:23.236 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.236 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC pong
2019-09-04 11:35:23.240 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 4
2019-09-04 11:35:23.242 DEBUG testcase.TC pong                         - 1: if (count != 0)
2019-09-04 11:35:23.243 DEBUG testcase.TC pong                         - 1: test.pp.pong(count)
pong! 4
2019-09-04 11:35:23.244 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.pong is PASSED
2019-09-04 11:35:23.244 DEBUG testcase.TC pong                         - 2: callTestCase(findTestCase("TC ping"), ["count":count])
2019-09-04 11:35:23.302 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.302 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC ping
2019-09-04 11:35:23.306 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 4
2019-09-04 11:35:23.307 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:23.307 DEBUG testcase.TC ping                         - 1: (count++)
2019-09-04 11:35:23.307 DEBUG testcase.TC ping                         - 2: test.pp.ping(count)
ping! 5
2019-09-04 11:35:23.308 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.ping is PASSED
2019-09-04 11:35:23.308 DEBUG testcase.TC ping                         - 3: callTestCase(findTestCase("TC pong"), ["count":count])
2019-09-04 11:35:23.365 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.365 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC pong
2019-09-04 11:35:23.368 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 5
2019-09-04 11:35:23.369 DEBUG testcase.TC pong                         - 1: if (count != 0)
2019-09-04 11:35:23.370 DEBUG testcase.TC pong                         - 1: test.pp.pong(count)
pong! 5
2019-09-04 11:35:23.370 INFO  k.k.c.m.CustomKeywordDelegatingMetaClass - test.pp.pong is PASSED
2019-09-04 11:35:23.371 DEBUG testcase.TC pong                         - 2: callTestCase(findTestCase("TC ping"), ["count":count])
2019-09-04 11:35:23.427 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.427 INFO  c.k.katalon.core.main.TestCaseExecutor   - CALL Test Cases/TC ping
2019-09-04 11:35:23.431 INFO  c.k.katalon.core.main.TestCaseExecutor   - count = 5
2019-09-04 11:35:23.432 DEBUG testcase.TC ping                         - 1: if (count < 5)
2019-09-04 11:35:23.432 DEBUG testcase.TC ping                         - 2: else
2019-09-04 11:35:23.432 DEBUG testcase.TC ping                         - 1: println("done!")
done!
2019-09-04 11:35:23.435 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC ping
2019-09-04 11:35:23.435 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.441 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC pong
2019-09-04 11:35:23.441 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.442 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC ping
2019-09-04 11:35:23.442 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.443 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC pong
2019-09-04 11:35:23.443 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.444 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC ping
2019-09-04 11:35:23.444 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.445 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC pong
2019-09-04 11:35:23.445 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.446 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC ping
2019-09-04 11:35:23.446 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.447 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC pong
2019-09-04 11:35:23.447 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.448 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC ping
2019-09-04 11:35:23.448 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.449 INFO  c.k.katalon.core.main.TestCaseExecutor   - END CALL Test Cases/TC pong
2019-09-04 11:35:23.449 INFO  c.k.katalon.core.main.TestCaseExecutor   - --------------------
2019-09-04 11:35:23.450 INFO  c.k.katalon.core.main.TestCaseExecutor   - END Test Cases/TC ping

Of-course, you can do the ping-pong other ways, e.g from a single testcase and making the keywords to call each-other, to avoid the nested log view result … or other solution i don’t have in mind right now.
Hope it helped. Have fun!

Warning! this method is most probably resource hungry, since each iteration will add again the objects in memory. Use it with caution!

Ibus · September 4, 2019, 10:00am

@ThanhTo i saw you liked it so … you can use this method for a quick perfmance test for new releases. do it with 1000 iterations and watch on the CPU and RAM usage. condition is to always use the same, or similar, hardware resources. it can be a quick way to tell you if a new version has improvements or is performing worse, wrt to hw usage. total running time may be also a good indicator.

ThanhTo · September 4, 2019, 10:20am

Hi @Ibus

We are already using a more sophisticated method to benchmark the performance. But I would keep this in mind in case it can expose a different type of problem in performance.

Cheers !

Ibus · September 4, 2019, 2:45pm

@ThanhTo i am positive you have more complex benchmarks, thats why mostly i see improvements with every new release. that’s why i mentioned ‘quick’ … like in ‘smoke test’.
keep up the good job!

jai8.jacob · September 5, 2019, 1:34am

@Ibus Thanks for getting back with a detailed response. Appreciate it. I’ve tried following it somewhat similar to how you’ve mentioned but I am still facing some issues running the POC.

From the image above, I’ve numbered the parts to which I need your attention to,

I’ve created two TCs under a folder named pingPongPOC named “Ping POC” and “Pong POC”.
Created a package under Keywords named “pingPongPOC” inside which is the ‘pp.groovy’ file that contains the two keywords that print the count values.
This is how my “Ping POC” file looks like.
This is how my “pp.grrovy” file looks like.
Error that I am getting.

I am assuming the way I can calling the keywords seems to be wrong(syntax error). Few variations I’ve tried are,

pingPongPOC.pp.ping(count)

and

pp.ping(count)

But both returns witch the same “No Such property error”.

Console:

2019-09-05 11:28:16.607 INFO c.k.katalon.core.main.TestCaseExecutor - --------------------
2019-09-05 11:28:16.610 INFO c.k.katalon.core.main.TestCaseExecutor - START Test Cases///pingPongPOC/Ping POC
2019-09-05 11:28:16.833 INFO c.k.katalon.core.main.TestCaseExecutor - (Default) count = 0
2019-09-05 11:28:17.108 ERROR c.k.katalon.core.main.TestCaseExecutor - Test Cases///pingPongPOC/Ping POC FAILED.
Reason:
groovy.lang.MissingPropertyException: No such property: pp for class: pingPongPOC.Script1567643841033
at pingPongPOC.Script1567643841033.run(Script1567643841033.groovy:25)
at com.kms.katalon.core.main.ScriptEngine.run(ScriptEngine.java:194)
at com.kms.katalon.core.main.ScriptEngine.runScriptAsRawText(ScriptEngine.java:119)
at com.kms.katalon.core.main.TestCaseExecutor.runScript(TestCaseExecutor.java:337)
at com.kms.katalon.core.main.TestCaseExecutor.doExecute(TestCaseExecutor.java:328)
at com.kms.katalon.core.main.TestCaseExecutor.processExecutionPhase(TestCaseExecutor.java:307)
at com.kms.katalon.core.main.TestCaseExecutor.accessMainPhase(TestCaseExecutor.java:299)
at com.kms.katalon.core.main.TestCaseExecutor.execute(TestCaseExecutor.java:233)
at com.kms.katalon.core.main.TestCaseMain.runTestCase(TestCaseMain.java:114)
at com.kms.katalon.core.main.TestCaseMain$runTestCase$0.call(Unknown Source)
at TempTestCase1567646895099.run(TempTestCase1567646895099.groovy:21)

2019-09-05 11:28:17.236 INFO c.k.katalon.core.main.TestCaseExecutor - END Test Cases///pingPongPOC/Ping POC

What am I doing wrong?

Ibus · September 5, 2019, 4:50am

@jai8.jacob you have a mistake there:

in the testcases, you don’t have to put the ‘package’ line.
also, expand the import line and make sure you are not missing the two needed

import static com.kms.katalon.core.testcase.TestCaseFactory.findTestCase
import com.kms.katalon.core.webui.keyword.WebUiBuiltInKeywords as WebUI

LE. in the package, expand the imports and make sure you are not missing this one:

import com.kms.katalon.core.annotation.Keyword

LLE: you will have to update also the code for ‘findTestCase’. since you have your testcases organized in a folder, you have to give the path to them, see:

albert.vu · August 8, 2022, 7:40am