How to tell pytest-xdist to run tests from one folder sequencially and the rest in parallel?

2024/9/16 23:35:59

Imagine that I have test/unit/... which are safe to run in parallel and test/functional/... which cannot be run in parallel yet.

Is there an easy way to convince pytest to run the functional ones sequentially? Consider that we are talking about a big number of tests so altering each test function/method would be very noisy.

At this moment we run tests with marker filters so mainly we run them separated. Still, I am looking for a solution for removing the need to run them separated.

Answer

You can implement your own scheduler that acts like load or loadscope, depending on what module the test is defined in. Example:

from xdist.scheduler.loadscope import LoadScopeSchedulingclass MyScheduler(LoadScopeScheduling):def _split_scope(self, nodeid):if 'test/functional' in nodeid:return 'functional-tests'return nodeiddef pytest_xdist_make_scheduler(config, log):return MyScheduler(config, log)

All tests under test/functional will be grouped under the same node functional-tests (and thus run in the same worker), the rest will be run in parallel as usual.

https://en.xdnf.cn/q/73040.html

Related Q&A

PyPDF4 - Exported PDF file size too big

I have a PDF file of around 7000 pages and 479 MB. I have create a python script using PyPDF4 to extract only specific pages if the pages contain specific words. The script works but the new PDF file,…

Jupyter install fails on Mac

Im trying to install Jupyter on my Mac (OS X El Capitan) and Im getting an error in response to:sudo pip install -U jupyterAt first the download/install starts fine, but then I run into this:Installing…

Python Error Codes are upshifted

Consider a python script error.pyimport sys sys.exit(3)Invokingpython error.py; echo $?yields the expected "3". However, consider runner.pyimport os result = os.system("python error.py&…

Running dozens of Scrapy spiders in a controlled manner

Im trying to build a system to run a few dozen Scrapy spiders, save the results to S3, and let me know when it finishes. There are several similar questions on StackOverflow (e.g. this one and this oth…

How to merge two DataFrame columns and apply pandas.to_datetime to it?

Im learning to use pandas, to use it for some data analysis. The data is supplied as a csv file, with several columns, of which i only need to use 4 (date, time, o, c). Ill like to create a new DataFr…

Breaking a parent function from within a child function (PHP Preferrably)

I was challenged how to break or end execution of a parent function without modifying the code of the parent, using PHPI cannot figure out any solution, other than die(); in the child, which would end …

Get absolute path of caller file

Say I have two files in different directories: 1.py (say, in C:/FIRST_FOLDER/1.py) and 2.py (say, in C:/SECOND_FOLDER/2.py).The file 1.py imports 2.py (using sys.path.insert(0, #path_of_2.py) followed,…

Pandas dataframe to excel gives file is not UTF-8 encoded

Im working on lists that I want to export into an Excel file.I found a lot of people advising to use pandas.dataframe so thats what I did. I could create the dataframe but when I try to export it to Ex…

Finding complex roots from set of non-linear equations in python

I have been testing an algorithm that has been published in literature that involves solving a set of m non-linear equations in both Matlab and Python. The set of non-linear equations involves input v…

Python on Raspberry Pi user input inside infinite loop misses inputs when hit with many

I have a very basic parrot script written in Python that simply prompts for a user input and prints it back inside an infinite loop. The Raspberry Pi has a USB barcode scanner attached for the input.wh…