Florian Bruhin shared a pytest tip on X \/ Twitter the other day regarding command line options that pytest accepts and that control how pytest handles failed tests.\nHe shared the following very<\/em> illustrative diagram:<\/p>\n I think the diagram is so good that I could essentially stop the article here and we all would've understood the options.\nBut I'll be just a little bit more verbose and explain how each option works.<\/p>\n Create the following file If you run the tests with The option Note how pytest says we'll pick up from here when we rerun the tests.\nLet us go ahead and fix our first test:<\/p>\n We rerun pytest with Note that pytest says that we passed one test ( If we attempt to fix it (but still fail), when we rerun pytest with Now, we don't even run Let's now fix Rerunning with Now, if we clear the pytest cache with Today I learned how to use named tuples to improve readability and flexibility of test parametrisations in pytest.<\/p>\n\n Today I attended a pytest tutorial<\/a> at PyCon DE 2024<\/a> and I learned a couple of really neat tricks to improve parametrised tests.<\/p>\n Let's take a look at them!<\/p>\n One quick tip is that you can stack For example, if Running this test, you get the following output:<\/p>\n With this pattern you cannot specify a test output for each single test, so this will only work if you have a different way of computing the result or if the expected result is always the same.<\/p>\n For tests where you must specify the expected result but you also want to test all permutations of certain parameters, you can use the REPL and You can add a lot of flexibility and readability in your parametrised tests if you use named tuples1<\/a><\/sup>.\nInstead of listing tuples with plenty of different values, we can create a named tuple to wrap around those tuples, which then lets us use named parameters and<\/em> default arguments in our parametrisation cases!<\/p>\n As an example, consider a test that's parametrised in the conventional way:<\/p>\n If you create a named tuple, you could use the names of the parameters to help disambiguate the two flags, for example, and you could also add default values:<\/p>\n This does add more code to your tests, so it's not really reasonable to do it for every single parametrisation you'll create, but it might come in handy for more complex parametrisations.\nI really liked this...<\/p>","summary":"Today I learned how to use named tuples to improve readability and flexibility of test parametrisations in pytest.","date_modified":"2025-09-06T19:56:10+02:00","tags":["modules","productivity","programming","pytest","python","testing"],"image":"\/user\/pages\/02.blog\/04.til\/095.better-test-parametrisation-in-pytest\/thumbnail.webp"},{"title":"TIL #094 \u2013 pytest.raises: parameter match","date_published":"2024-04-22T16:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/pytest.raises-parameter-match","url":"https:\/\/mathspp.com\/blog\/til\/pytest.raises-parameter-match","content_html":" Today I learned about the parameter Today I attended a pytest tutorial<\/a> at PyCon DE 2024<\/a> and very early in the tutorial I learned about the parameter The parameter The initial example given was this:<\/p>\n This function raises a So, when writing a test to make sure that The trainer also put forward that this parameter Yesterday I spent the whole day tryint to patch a module global.\nThis is what I ended up with.<\/p>\n\n Suppose you have a small project that has an April Fool's joke set up:<\/p>\n Now, suppose you want to test your April Fool's joke with (I'm sparing you from hearing about all of the embarrissingly dumb things I tried before getting to this point!)<\/p>\n If you run this test with What's happening, then?\nTake a moment to think about it...<\/p>\n What's happening is that when I import After banging my head against the wall for a while, I realised I could tweak my code to work with this testing approach!\nAll I had to do was use Now, if you run your test it will pass and the mocking of the variable will have succeeded!<\/p>\n The reason this works is that now we're accessing You could also patch I'm not claiming this is the best solution<\/em> to this problem but it's certainly a solution that works<\/em>.\nIf you have better ideas that have more or less the same complexity, let me know!<\/p>\n For reference, this is a simplification of a \u201creal\u201d issue I had when trying to add tests to this Textual PR of mine<\/a>.<\/p>","summary":"Today I learned how to patch module globals with pytest.","date_modified":"2025-07-23T16:49:02+02:00","tags":["programming","pytest","python","testing"],"image":"\/user\/pages\/02.blog\/04.til\/090.patching-module-globals-with-pytest\/thumbnail.webp"},{"title":"TIL #063 \u2013 skip tests on Microsoft Windows","date_published":"2023-06-12T00:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/skip-tests-on-microsoft-windows","url":"https:\/\/mathspp.com\/blog\/til\/skip-tests-on-microsoft-windows","content_html":" Today I learned how to skip tests on Microsoft Windows in In For example, the test below will never run:<\/p>\n Instead of running the test, However, the decorator If you want to skip a For example, the test below will only run if the current Python version is 3.7 or higher:<\/p>\n The decorators We can do a similar thing if we only want a test to run under Windows.\nOr if we do not<\/em> want a test to run under Windows.<\/p>\n If you want to skip a test if you are running on Windows, you would mark it with For example, the test below will not run if the current operating system is Microsoft Windows:<\/p>\n If you want to skip a whole test file, you can assign the result of There are several plausible reasons why you'd want to skip a couple of tests in “real life”.<\/p>\n For example, you might want to skip some tests in certain versions of Python if you are trying to use functionality that was introduced in a later version of Python.\nThis could also apply to specific versions of certain dependencies.<\/p>\n Another example, and the reason that led to me learning this, was when working on Textual<\/a>: I needed to skip a specific test if I was not<\/em> running under Microsoft Windows because I had to implement a function that would only be called under Windows and that would use Windows-specific APIs...\nSo, even if I wanted to run the test on other operating systems, the test could never pass because those operating systems...<\/p>","summary":"Today I learned how to skip tests on Microsoft Windows in `pytest`.","date_modified":"2025-07-23T16:49:02+02:00","tags":["modules","programming","pytest","python"],"image":"\/user\/pages\/02.blog\/04.til\/063.skip-tests-on-microsoft-windows\/thumbnail.webp"}]}
![\"Diagram](\"\/user\/pages\/02.blog\/04.til\/105.pytest-selection-arguments-for-failing-tests\/_diagram.webp\" "\\"Diagram")
--maxfail<\/code><\/a><\/h3>\ntest.py<\/code> with 4 failing tests:<\/p>\ndef test_1():\n assert False\n\ndef test_2():\n assert False\n\ndef test_3():\n assert False\n\ndef test_4():\n assert False<\/code><\/pre>\npytest test.py<\/code> you get 4 failing tests. Duh.\nIf you use the option --maxfail=n<\/code>, pytest will run until completion or until it finds n<\/code> failures.\nRunning with pytest test.py --maxfail=2<\/code> stops after test_2<\/code> failed:<\/p>\nFAILED test.py::test_1 - assert False\nFAILED test.py::test_2 - assert False\n!!!!!!!!!! stopping after 2 failures !!!!!!!!!!!\n============== 2 failed in 0.04s ===============<\/code><\/pre>\n--stepwise<\/code><\/a><\/h3>\n--stepwise<\/code> stop at each failure and every time you rerun, it picks up from where it left off.\nStarting with the file from the previous example, if you run pytest test.py --stepwise<\/code>, we stop immediately at the first failure:<\/p>\nFAILED test.py::test_1 - assert False\n! Interrupted: Test failed, continuing from this test next run. !\n============== 1 failed in 0.08s ===============<\/code><\/pre>\ndef test_1():\n assert True<\/code><\/pre>\npytest test.py --stepwise<\/code> and we go up to test 2:<\/p>\ntest.py .F\nFAILED test.py::test_2 - assert False\n! Interrupted: Test failed, continuing from this test next run. !\n========= 1 failed, 1 passed in 0.08s ==========<\/code><\/pre>\ntest_1<\/code>, which was fixed) and that we failed one test (test_2<\/code>).<\/p>\npytest test.py --stepwise<\/code> we start from test 2 and it will fail again:<\/p>\nFAILED test.py::test_2 - assert False\n! Interrupted: Test failed, continuing from this test next run. !\n======= 1 failed, 1 deselected in 0.08s ========<\/code><\/pre>\ntest_1<\/code> because we're already past it.\nWe tried rerunning test_2<\/code> but it still failed.<\/p>\ntest_2<\/code>:<\/p>\ndef test_2():\n assert True<\/code><\/pre>\npytest test.py --stepwise<\/code> skips test_1<\/code>, retries test_2<\/code> and goes up to test_3<\/code>:<\/p>\nFAILED test.py::test_3 - assert False\n! Interrupted: Test failed, continuing from this test next run. !\n== 1 failed, 1 passed, 1 deselected in 0.08s ===<\/code><\/pre>\nrm -rf .pytest_cache<\/code> and rerun with pytest test.py --stepwise<\/code>, we see that we run the first three tests, stopping at test_3<\/code> because that...<\/p>","summary":"Today I learned about 5 useful pytest options that let me control what tests to run with respect to failing tests.","date_modified":"2025-10-20T22:34:56+02:00","tags":["productivity","programming","python","pytest"],"image":"\/user\/pages\/02.blog\/04.til\/105.pytest-selection-arguments-for-failing-tests\/thumbnail.webp"},{"title":"TIL #095 \u2013 better test parametrisation in pytest","date_published":"2024-04-22T17:00:00+02:00","id":"https:\/\/mathspp.com\/blog\/til\/better-test-parametrisation-in-pytest","url":"https:\/\/mathspp.com\/blog\/til\/better-test-parametrisation-in-pytest","content_html":"Better test parametrisation in pytest<\/a><\/h2>\n
Stacking the
parametrize<\/code> mark<\/a><\/h3>\nparametrize<\/code> to run all permutations of different parameters.<\/p>\ncalc<\/code> is a function that does basic arithmetic operations, the test below will test 9 different additions:<\/p>\nimport pytest\n\n@pytest.mark.parametrize(\"a\", [-3, 0, 5])\n@pytest.mark.parametrize(\"b\", [-8, 0, 42])\ndef test_calc_add(a, b):\n assert calc(a, b, \"+\") == a + b<\/code><\/pre>\ntest_calc.py::test_calc_add[-8--3] PASSED [ 11%]\ntest_calc.py::test_calc_add[-8-0] PASSED [ 22%]\ntest_calc.py::test_calc_add[-8-5] PASSED [ 33%]\ntest_calc.py::test_calc_add[0--3] PASSED [ 44%]\ntest_calc.py::test_calc_add[0-0] PASSED [ 55%]\ntest_calc.py::test_calc_add[0-5] PASSED [ 66%]\ntest_calc.py::test_calc_add[42--3] PASSED [ 77%]\ntest_calc.py::test_calc_add[42-0] PASSED [ 88%]\ntest_calc.py::test_calc_add[42-5] PASSED [100%]<\/code><\/pre>\nitertools.product<\/code> to programmatically generate all the tests instead of having to write them down by hand.<\/p>\nParametrising with named tuples<\/a><\/h3>\n
import pytest\n\n@pytest.mark.parametrize(\n [\"a\", \"b\", \"op\", \"result\", \"flag1\", \"flag2\"],\n [\n (10, 15, \"+\", 25, True, False),\n (10, 15, \"-\", -5, True, False),\n (3, 18, \"*\", 54, False, False),\n (18, 3, \"\/\", 6, True, False),\n (5, 0, \"\/\", None, True, True),\n ]\n)\ndef test_something(a, b, op, result, flag1, flag2):\n # do stuff with a, b, op, ...\n ...<\/code><\/pre>\nfrom typing import NamedTuple\nimport pytest\n\nclass CaseParam(NamedTuple):\n a: int\n b: int\n op: str\n result: int\n flag1: bool = True\n flag2: bool = False\n\n@pytest.mark.parametrize(\n [\"a\", \"b\", \"op\", \"result\", \"flag1\", \"flag2\"],\n [\n CaseParam(10, 15, \"+\", 25),\n CaseParam(10, 15, \"-\", -5),\n CaseParam(3, 18, \"*\", 54, flag1=False),\n CaseParam(18, 3, \"\/\", 6),\n CaseParam(5, 0, \"\/\", None, flag2=True),\n ]\n)\ndef test_something(a, b, op, result, flag1, flag2):\n # do stuff with a, b, op, ...\n ...<\/code><\/pre>\nmatch<\/code> used in pytest.raises<\/code>.<\/p>\n\npytest.raises<\/code>: parameter match<\/code><\/a><\/h2>\nmatch<\/code> that you can use with pytest.raises<\/code>.<\/p>\nmatch<\/code> accepts a regex pattern that is tested against the error message and your test only passes if the error message matches the regex.<\/p>\ndef parse_pos_number(s: str) -> int:\n n = int(s)\n if n < 0:\n raise ValueError(\"No negativity allowed!\")\n return n<\/code><\/pre>\nValueError<\/code> if you pass it a negative value but it will also raise a (different) ValueError<\/code> if you pass it a string that doesn't look like an integer:<\/p>\nparse_pos_number(\"-1\") # ValueError: No negativity allowed!\nparse_pos_number(\"a\") # ValueError: invalid literal for int() with base 10: 'a'<\/code><\/pre>\nparse_pos_number<\/code> fails when giving it a negative number, we should also ensure that the ValueError<\/code> is \u201cthe correct one\u201d.\nThe test would look like this:<\/p>\nimport pytest\n\ndef test_with_negative_number():\n with pytest.raises(ValueError, match=\"No negativity allowed!\"):\n parse_pos_number(\"-1\")<\/code><\/pre>\nmatch<\/code> can be quite useful if you have a (custom) error that is raised in multiple situations.\nAdding match<\/code> to your tests will also test your error messages and it acts as a \u201cdocumentation\u201d test.<\/p>","summary":"Today I learned about the parameter `match` used in `pytest.raises`.","date_modified":"2025-07-23T16:49:02+02:00","tags":["modules","programming","pytest","python","testing"],"image":"\/user\/pages\/02.blog\/04.til\/094.pytest.raises-parameter-match\/thumbnail.webp"},{"title":"TIL #090 \u2013 patching module globals with pytest","date_published":"2024-01-25T09:30:00+01:00","id":"https:\/\/mathspp.com\/blog\/til\/patching-module-globals-with-pytest","url":"https:\/\/mathspp.com\/blog\/til\/patching-module-globals-with-pytest","content_html":"Patching module globals with
pytest<\/code><\/a><\/h2>\n## constants.py\n\nAPRIL_FOOLS = False<\/code><\/pre>\n## useful_code.py\n\nfrom constants import APRIL_FOOLS\n\ndef add(a, b):\n if APRIL_FOOLS:\n result = a - b\n else:\n result = a + b\n return result<\/code><\/pre>\npytest<\/code>.\nOne thing I found online was that you could use the fixture monkeypatch<\/code> and its method setattr<\/code>, something like this:<\/p>\n## test_useful_code.py\n\nimport pytest\n\nimport constants\nimport useful_code\n\ndef test_add_on_april_fools(monkeypatch):\n monkeypatch.setattr(constants, \"APRIL_FOOLS\", True)\n assert useful_code.add(10, 5) == 5<\/code><\/pre>\npytest test_useful_code.py<\/code> you get a failure:<\/p>\nFAILED test_useful_code.py::test_add_on_april_fools - assert 15 == 5<\/code><\/pre>\nuseful_code<\/code> into the testing file, that module will then run the line from constants import APRIL_FOOLS<\/code>, which sets the variable APRIL_FOOLS<\/code> inside the namespace of the module useful_code<\/code>, which is separate<\/em> from the namespace of the module constants<\/code>!<\/p>\nconstants.APRIL_FOOLS<\/code> instead of using APRIL_FOOLS<\/code> directly:<\/p>\n## useful_code.py\n\nimport constants\n\ndef add(a, b):\n if constants.APRIL_FOOLS:\n result = a - b\n else:\n result = a + b\n return result<\/code><\/pre>\nAPRIL_FOOLS<\/code> via the constants<\/code> namespace, so when we patch the value of APRIL_FOOLS<\/code> in constants<\/code>, that patched value will be visible when the constant is used elsewhere.<\/p>\nAPRIL_FOOLS<\/code> directly in the module useful_code<\/code>, but if you have a constant that is used in many different modules, it is much easier (and better) to patch it once<\/em> where it is defined versus patching it many times wherever it is imported.<\/p>\npytest<\/code>.<\/p>\n\npytest.mark.skip<\/code><\/a><\/h2>\npytest<\/code>, you can skip a specific test by using the decorator pytest.mark.skip<\/code>.<\/p>\nimport pytest\n\n@pytest.mark.skip\ndef test_yeah():\n assert False<\/code><\/pre>\npytest<\/code> shows it was skipped:<\/p>\n![\"\"](\"\/blog\/til\/skip-tests-on-microsoft-windows\/.\/_skip.webp\")
<\/p>\npytest.mark.skip<\/code> will always<\/em> skip your test.\nSometimes, you only want to skip under certain circumstances.<\/p>\nHow to conditionally skip a test in
pytest<\/code>?<\/a><\/h2>\npytest<\/code> test conditionally, you need to use the decorator pytest.mark.skipif<\/code>.<\/p>\nimport pytest\nimport sys\n\n@pytest.mark.skipif(sys.version_info < (3, 7))\ndef test_foo():\n assert True<\/code><\/pre>\nskip<\/code> and skipif<\/code> also accept a keyword argument reason<\/code> that you can use to give more information as to why you are skipping a given test:<\/p>\nimport pytest\nimport sys\n\n@pytest.mark.skip(reason=\"We skip because this always fails.\")\ndef test_yeah():\n assert False\n\n@pytest.mark.skipif(sys.version_info < (3, 7), reason=\"Only works on 3.7+.\")\ndef test_foo():\n assert True<\/code><\/pre>\nHow to skip a test on Microsoft Windows<\/a><\/h2>\n
skipif<\/code> and then you check if the platform is Windows.<\/p>\nimport pytest\nimport sys\n\n@pytest.mark.skipif(\n sys.platform ==\"win32\", reason=\"Windows doesn't have what it takes.\"\n)\ndef test_something_not_on_windows():\n assert ...<\/code><\/pre>\nHow to skip a whole test file<\/a><\/h2>\n
skipif<\/code> to the global variable pytestmark<\/code>.\nFor example, the file below would only run its tests if you were running on Windows and specifically with Python 3.10:<\/p>\nimport pytest\nimport sys\n\npytestmark = pytest.mark.skipif(\n sys.platform == \"win32\" and sys.version_info == (3, 10)\n)\n\ndef test_one():\n assert 1 == 1\n\ndef test_two():\n assert int(\"2\") == 2\n\ndef test_three():\n assert str(3) == \"3\"<\/code><\/pre>\nWhy would you skip some tests?<\/a><\/h2>\n