Stop Execution after some Failures

If we need to stop execution of the test suite after N number of failures then we need to pass a argument --maxfail =’number’ along with the pytest command.

pytest --maxfail = 1

Suppose when we are executing a test suite, Due to some issues most of the test cases are failing in the test suite then there is no point to wait until execution completes. We need to stop the execution after N number of tests fails at such a point we need to pass the argument as --maxfail=’number’ .

Lets see the difference once. In the below example we are not passing the --maxfail argument and executing the script.

Here it will execute all the methods, in the below example we are failing two methods to show the difference.



test_StopTestSuiteConcept.py

def test_methodA():
    print("This is method A")
    assert 1 == 2


def test_methodB():
    print("This is method B")
    assert 1 == 2


def test_methodC():
    print("This is method C")


def test_methodD():
    print("This is method D")

Run the command to execute the pytest file.

pytest -v -s test_StopTestSuiteConcept.py


OutPut data

admin@admin-MacBook stoptestsconcept % pytest -v -s            
============================================================================ test session starts ============================================================================
platform darwin -- Python 3.8.3, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
cachedir: .pytest_cache
rootdir: /Users/admin/Documents/Skill2Lead/Tutorials/pytest/stoptestsconcept
plugins: rerunfailures-9.0, allure-pytest-2.8.16, ordering-0.6
collected 4 items                                                                                                                                                           

test_StopTestSuiteConcept.py::test_methodA This is method A
FAILED
test_StopTestSuiteConcept.py::test_methodB This is method B
FAILED
test_StopTestSuiteConcept.py::test_methodC This is method C
PASSED
test_StopTestSuiteConcept.py::test_methodD This is method D
PASSED

========================================================================== short test summary info ==========================================================================
FAILED test_StopTestSuiteConcept.py::test_methodA - assert 1 == 2
FAILED test_StopTestSuiteConcept.py::test_methodB - assert 1 == 2
======================================================================== 2 failed, 2 passed in 0.10s ========================================================================
admin@admin-MacBook stoptestsconcept %

Now let us execute the script using --maxfail argument.

Run the command to execute the pytest file.

pytest -v -s test_StopTestSuiteConcept.py --maxfail=1


OutPut data


admin@admin-MacBook stoptestsconcept % pytest -v -s --maxfail=1 
============================================================================ test session starts ============================================================================
platform darwin -- Python 3.8.3, pytest-5.4.3, py-1.8.1, pluggy-0.13.1 -- /Library/Frameworks/Python.framework/Versions/3.8/bin/python3.8
cachedir: .pytest_cache
rootdir: /Users/admin/Documents/Skill2Lead/Tutorials/pytest/stoptestsconcept
plugins: rerunfailures-9.0, allure-pytest-2.8.16, ordering-0.6
collected 4 items                                                                                                                                                           

test_StopTestSuiteConcept.py::test_methodA This is method A
FAILED

================================================================================= FAILURES ==================================================================================
_______________________________________________________________________________ test_methodA ________________________________________________________________________________

    def test_methodA():
        print("This is method A")
>       assert 1 == 2
E       assert 1 == 2
E         +1
E         -2

test_StopTestSuiteConcept.py:3: AssertionError
========================================================================== short test summary info ==========================================================================
FAILED test_StopTestSuiteConcept.py::test_methodA - assert 1 == 2
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
============================================================================= 1 failed in 0.09s =============================================================================
admin@admin-MacBook stoptestsconcept %

Here you can see that execution has stopped immediately after one test case has failed because we have passed ‘1’ as an argument with --maxfail.

Prev
Next