Regression: passes watch dir to py.test about pytest-watch HOT 11 CLOSED

This was done in ede80ff.

from pytest-watch.

Yeah, this was intended. (Was motivated by that commit and also to be consistent with --ignore being forwarding to pytest from #48.)

Thinking more about it though. Here's the use cases I was thinking of:

1. Run from subdirectory, watching and collecting tests from all subdirectories:

   • Formerly ptw myproject -- myproject
   • Currently ptw myproject

2. Ignore unrelated dirs from watch and test collection (due to large trees slowing things down):

   • Formerly ptw --ignore tmp -- --ignore tmp
   • Currently ptw --ignore tmp

   Or from a subdirectory:

   • Formerly ptw myproject --ignore myproject/tmp -- myproject --ignore myproject/tmp
   • Currently ptw myproject --ignore myproject/tmp

   (This actually was never implemented, which I forgot about! Just opened #56 to address this.)

3. Include watched directories without collecting tests from them

   • Formerly ptw myproject mytests -- mytests
   • Currently not possible (ptw myproject mytests -- --ignore myproject doesn't work)

4. Include test collection from a directory without watching them

   • Formerly ptw myproject -- myproject mytests
   • Currently ptw myproject -- mytests

(1)-(3) re-runs the tests when changing either source code or test files.

(4) only re-runs when changing source code, which doesn't seem that useful.

So the issue is that (3) isn't possible now. Is it really a problem though? It almost seems like a pre-optimization to try to exclude test collection from a watched directory. Remember, watchdog also has to crawl through a huge directory, so you'll have delay for huge project directories regardless. A better fix might be to exclude the things that aren't testable from both (static and template files for websites,vendor directories, etc.)

Or is there another case I'm leaving out?

from pytest-watch.

I am using testpaths = path/to/tests, so there is no need to pass path/to/tests to py.test.

So with ptw path, path would be watched, but only path/to/tests used to collect tests.

If you want to keep the current behavior, then the testpaths option should be looked at and only if it's not used the watch directory should be passed explicitly to py.test?

from pytest-watch.

So with ptw path, path would be watched, but only path/to/tests used to collect tests.

If you pass arguments to py.test, does that overwrite testpaths?

from pytest-watch.

Btw, I'm also considering having pytest-watch be usable as a standard pytest plugin. Making the command line arguments more consistent with pytest was a first step. (Just opened #57 to dedicate to that discussion.)

from pytest-watch.

If you pass arguments to py.test, does that overwrite testpaths?

Yes.

from pytest-watch.

Ah, that's a shame.

The idea is that ptw dir is to tell both watchdog and pytest to start their search from there. Some more questions for you:

• Are you able to run ptw from your project root instead of specifying where to watch and where to collect tests? If not, is it because:
  • Your tests are located outside of the project?
  • It's taking too long to boot?
• Are you able to run cd dir && ptw instead?

from pytest-watch.

Ah, that's a shame.

Only in this context, right? Because otherwise it's quite nice, isn't it?

While I could workaround this behaviour change, I'd rather not do so.. :)

Are you able to run ptw from your project root instead of specifying where to watch and where to collect tests?

Yes, but I'd like to avoid scanning everything (build dir, venv etc). Even the project's source can be skipped. That's why I like testpaths.

Are you able to run cd dir && ptw instead?

Yes.

What speaks against using the same behaviour regarding testpaths as py.test though?

from pytest-watch.

Only in this context, right? Because otherwise it's quite nice, isn't it?

Yeah. More like it's a shame that I overlooked that option / missed your use case. Was mostly asking these questions to make sure I didn't miss something else.

Thinking more about it, I suppose (3) is important after all to keep test runs lightning fast. Perhaps a new named argument could be used for watching without collecting tests. This would continue to make sense if pytest-watch was used as a pytest plugin.

As for the case you've pointed out, what do you think of having an equivalent config option pytest-watch? Say, watchpaths under the [pytest-watch] category. Like testpaths, it wouldn't have any effect when passing in <dir> arguments. Then you could run ptw with no arguments and fully customize what paths to watch and what paths to collect tests from.

from pytest-watch.

@joeyespo

As for the case you've pointed out, what do you think of having an equivalent config option pytest-watch?

Sounds good to me!

from pytest-watch.

Just opened #61 to address this. Closing as per this comment, but feel free to re-open if you don't think that'll be enough to help you.

from pytest-watch.