Fail tests that take too long to run

Last update: Nov 27, 2022

Overview

Project Status: Active — The project has reached a stable, usable state and is being actively developed.

GitHub | PyPI | Issues

pytest-fail-slow is a pytest plugin for making tests fail that take too long to run. It adds a --fail-slow DURATION command-line option to pytest that causes any otherwise-passing tests that run for longer than the given duration to be marked as failures. Note that slow tests will still be run to completion; if you want them to instead be stopped early, use pytest-timeout.

A duration can be supplied to the --fail-slow option as either a bare floating-point number of seconds or as a floating-point number followed by one of the following units (case insensitive):

h, hour, hours
m, min, mins, minute, minutes
s, sec, secs, second, seconds
ms, milli, millisec, milliseconds
us, μs, micro, microsec, microseconds

If pytest-fail-slow marks a test as a failure, the output will include the test's duration and the duration threshold, like so:

________________________________ test_func ________________________________
Test passed but took too long to run: Duration 123.0s > 5.0s

Note: Only the durations for tests themselves are taken into consideration. If a test passes in less than the specified duration, but one or more fixture setups/teardowns take longer than the duration, the test will still be marked as passing.

Installation

pytest-fail-slow requires Python 3.6 or higher and pytest 6.0 or higher. Just use pip for Python 3 (You have pip, right?) to install it:

python3 -m pip install pytest-fail-slow

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations

To automate the generation and validation tests of COSE/CBOR Codes and it's base45/2D Code representations, a lot of data has to be collected to ensure the variance of the tests. This respository was established to collect a lot of different test data and related test cases of different member states in a standardized manner. Each member state can generate a folder in this section.