Looks like it doesn’t take too much work to get pytest working on our code
base, since it knows how to run unittest tests.

https://github.com/apache/spark/compare/master…nchammas:pytest
<https://github.com/apache/spark/compare/master...nchammas:pytest>

For example I was able to do this from that branch and it did the right
thing, running only the tests with string in their name:

python [pytest *]$ ../bin/spark-submit ./pytest-run-tests.py
./pyspark/sql/tests.py -v -k string

However, looking more closely at the whole test setup, I’m hesitant to work
any further on this.

My intention was to see if we could leverage pytest, tox, and other test
tools that are standard in the Python ecosystem to replace some of the
homegrown stuff we have. We have our own test dependency tracking code, our
own breakdown of tests into module-scoped chunks, and our own machinery to
parallelize test execution. It seems like it would be a lot of work to reap
the benefits of using the standard tools while ensuring that we don’t lose
any of the benefits our current test setup provides.

Nick

On Tue, Aug 15, 2017 at 3:26 PM Bryan Cutler [EMAIL PROTECTED]
<http://mailto:[EMAIL PROTECTED]> wrote:

This generally works for me to just run tests within a class or even a