Refactor how tests are run about phpunit HOT 10 CLOSED

Data providers should be run once. If that means that counting no longer works and we need to change how progress is displayed, so be it.

from phpunit.

I'm very excited to see this! We've had to increase our memory limit every so often to deal with test cases sticking around to the end. One thing you might not have considered yet is that by default any exceptions thrown during a test will have a reference to the test case itself in the backtrace.

As I've started working on a new Java project at my job I'm picking up the new JUnit features. One neat feature is the ability to add a TestDecorator to any test case using an annotation. This is how jMock checks its mocks at the end of each method, and a similar docblock annotation could be helpful for PHP tool integrators.

from phpunit.

Sebastian, this looks like a good move. It also looks like it will resolve the issues I reported in #261.

Can you confirm that the behaviour I mention there is not intentional and won't be somehow preserved by your refactor?

from phpunit.

I just did a quick hack where I have a single TestResult and I use it to execute on a loop of TestSuite-s which each correspond to a single file, so that I can deallocate the suite after each test file is executed. I saw in my test suite a reduction from 643MB to 431MB. So very excited to see this implemented better.

from phpunit.

Hello @sebastianbergmann could you please add more details to this issue?
It would be nice if you could create some kind of a document with list of features/requirements that are planned for 4.0 release for example:

• long_class_names -> namespaces
• parallel test execution
• etc., etc.

Basically I think that new 4.0 branch is a good moment to solve old problems and implement existing functionality in cleaner way and I hope I'll be able to help in coding it.

from phpunit.

Before I report this as a bug, as it seems to be a design issue. Running tests with --filter causes all data providers to run. For some reason a test was failing when running it by file name rather than with a filter, as it was influenced by a data provider.

from phpunit.

• Sort the tests according to @Depends annotations

What is the status of this?

from phpunit.

wouldn't it make sense to also delay the creation of DataProviderTestSuites when doing this? They'd still need to collect all the data beforehand with the proposed changes, keeping them rather large.

This would obviously need the data providers to either be run twice(i.e. one for counting, one later for the data) or allow for changes to the testcase maximum count once the provider is being processed.

from phpunit.

data providers can provide also a label for cases, which one can use as a value for tests filter he want to run,
this feature shall not be broken by accident

from phpunit.

Superseded by #3213.

from phpunit.