I’m working on developing some test cases for a proto website. Some of the test case ideas have been to intentionally have test cases fail in order to demonstrate the website is functioning properly. For example, the website would not accept a negative number in the amount field. To test it, we thought of building a test that entered a negative number expecting the test to fail.
The test cases are being run in a docker container and built by Jenkins. I’ve noticed that when a test case fails, the Jenkins build also fails. I would like that the test cases that are written to intentionally fail show pass when they fail because it functions as designed. In the example above, if the negative number isn’t accepted the test produces an error “negative number entered” and the test passes. But if the test doesn’t produce an error and passes then the site is not functioning properly.
If there isn’t a way to do this, how then can I test examples like the one mentioned above?
failed test case means that something is wrong with your application. Rather then changing output of TC arbitrary i would go for change of logic.
Login with wrong credential shows an error - Assert should be based on fact that user is not logged in rather that intentionally make it fall and failed TC considered as positive result.
I agree with Andrej… test cases shouldn’t fail. You expect the function/feature to fail, so make a test that passes when the error occurs, since you expect it to occur.
you should not write a test that you expect to fail.
I would reword this as:
To test it, we thought of building a test that entered a negative number expecting a message is displayed: e.g, “we don’t accept a negative number here” .
and you can write a test that asserts this positively.
Thank you for all the advice! They have been most helpful!
It is the perspective of how you look at things.
A test case fails because we indicate this, your example test does not fail, you are doing the right thing by displaying an error message, the test will fail if you enter an incorrect character.