Hi there,
Earlier today, I upgraded to 5.4 and tried to run my text suite collection from the console with the flag -executionProfile=develop (full command below). I’ve found that this is still using the default profile. Does anyone know why this is?
katalon -runMode=console -executionProfile=%profile% -projectPath=“C:\Jenkins\workspace\sharedspace\MobileEnterpriseTest\MobileEnterpriseTest.prj” -retry=0 -testSuiteCollectionPath=“Test Suites/MobileEnterprise-Android/LoginTests”
Thanks!
We will double check this once again.
Regards
Hi Vinh - checking in to see if there has been any update with this problem? This feature in 5.4 is a great step forward for us and we would love to be able to take advantage of it for integrating Katalon tests with Jenkins builds - please let me know if there’s any more information you need from me. Thank you.
Hi Vinh - checking again to see if there has been any update here. Earlier today, I updated everything to the 5.4.1 release and still - all tests run against ‘default’ when using the console mode execution. Thank you.
I’ve changed my profiles and now, system can’t find any GlobalVariables values that are defined in any of profiles. Error msg: “FAILED because (of) java.lang.NullPointerException: Cannot execute null+{”
If I define variables in a test case everything works fine.
Can you just pass in a profile’s name directly instead of using variable like that? Example -executionProfile=“staging”
Vinh Nguyen said:
Can you just pass in a profile’s name directly instead of using variable like that? Example -executionProfile=“staging”
I have tried this actually also with no luck. Is this featured designed only for Test Suites or should this also work with Test Suite Collections?
Micaela,
In the Test Suite Collection pane, each individual Test Sites are assigned with Execution Profiles. I am afraid you are not aware of this.
Your command line has -executionProfile switch:
katalon -runMode=console -executionProfile=%profile% ... -testSuiteCollectionPath="..."
I suppose that the -executionProfile switch is just ignored if -testSuiteCollectionPath is specified. Effectively your command line is the same as this:
katalon -runMode=console ... -testSuiteCollectionPath="..."
Then Katalon Studio would happily find the name of Exception Profile to apply to each Test Suites as defined in the above screenshot.
TestSuiteCollection_assigning_different_profiles_to_testsuites.png
kazurayam said:
Micaela,
In the Test Suite Collection pane, each individual Test Sites are assigned with Execution Profiles. I am afraid you are not aware of this.
Your command line has -executionProfile switch:
katalon -runMode=console -executionProfile=%profile% ... -testSuiteCollectionPath="..."I suppose that the -executionProfile switch is just ignored if -testSuiteCollectionPath is specified. Effectively your command line is the same as this:
katalon -runMode=console ... -testSuiteCollectionPath="..."Then Katalon Studio would happily find the name of Exception Profile to apply to each Test Suites as defined in the above screenshot.
Hi Kazurayam - thank you for the suggestion! Actually I am aware of the profiles set here, however I was under the impression that these profiles were overriden when another profile was passed in by command line. The reason we want to use this feature is to run the *same* test suite collection with different profiles at run time, so setting the profile ahead of time like this would unfortunately defeat the purpose. If your explanation is correct though, is there a way to completely unset the Profile in the test suite so that when the test suite ran it would be forced to use the profile from the command line? I’ve been looking for a way but have not found it. Thanks,
Micaela
If your explanation is correct though, is there a way to completely unset the Profile in the test suite so that when the test suite ran it would be forced to use the profile from the command line?
Katalon team,
It’s your turn. Please explain what is correct.
however I was under the impression that these profiles were overriden when another profile was passed in by command line
I agree. It is better.
kazurayam said:
Micaela,
In the Test Suite Collection pane, each individual Test Sites are assigned with Execution Profiles. I am afraid you are not aware of this.
Your command line has -executionProfile switch:
katalon -runMode=console -executionProfile=%profile% ... -testSuiteCollectionPath="..."I suppose that the -executionProfile switch is just ignored if -testSuiteCollectionPath is specified. Effectively your command line is the same as this:
katalon -runMode=console ... -testSuiteCollectionPath="..."Then Katalon Studio would happily find the name of Exception Profile to apply to each Test Suites as defined in the above screenshot.
That’s in the case of test suites. In my case Im having test case creation in progress and in order to check them with local and server data I use profiles, so the profile is chosen manually on runtime.
“executionProfile=“staging”” This one does not work, same error msg.
It looks like it does not recompile bin folder and GlobalVariable.class and keeps previous versions (\bin\lib\internal\GlobalVariable.class), thats why it keeps default profiles values. Simple fix is to delete bin folder. It works for me now.
For us this issue occurred when the project was created with one version of Katalon Studio and tried to be executed with different version. Issue was resolved once we tried to execute with same version.
We’re using 5.7.0 and seeing the -executionProfile parameter being ignored in favor of the per-test-suite profile configured in the test suite collection for each test suite.
The -executionProfile parameter correctly changes the profile for individual test suites, but not for the test suite collection.
Does anyone know if this is by design and if there is a workaround or coming fix?
Workaround:
When running test suite collections, I manually change the profile for the suites.
kazurayam said:
however I was under the impression that these profiles were overriden when another profile was passed in by command line
I agree. It is better.
I agree 100%.
When running in Katalon Studio, the framework should listen to the IDE.
When running from the command line, the framework should listen to the command line.
The clue is in the name - it’s a COMMAND.
Mate Mrse said:
Workaround:
When running test suite collections, I manually change the profile for the suites.
That’s a great idea…if we weren’t looking to use these in a semi-automated environment. Stopping to edit the test suite collections, or have different test suite collections per environment, isn’t feasible.
As already mentioned, the -executionProfile parametter it is considered only when using a TestSuite execution.
So one option will be to run a test suite instead of a collection.
If really a TestCollection is needed to be executed, with custom profile, a different approach can be made (I am using a similar technique for running some test cases with a custom generated data set).
The collection in fact it is simply an xml file (check the .ts in the TestSuites folder).
So, you can use any text parsing method to replace the default with anything you like (for this purpose, on a linux Jenkins executor, i simply use some *sed* commands in an shell step, before to actually start the test execution).
Otherwise, you can simply make as many TestCollections as needed, with already set profiles