Before/After not working about data-driven HOT 8 CLOSED

In case somebody is interested: the solution is to just use another test suite WITHIN the data-driven suite, which has it's own before/after methods. This way nice 'boxing' effect is working.
Example:

describe("Outer test suite", function() {
  before(function(done) {
    console.log('I run before the whole test suite');
  });
  beforeEach(function(done) {
    console.log('I run before each test of the whole test suite');
  });
  data-driven([{version: 'v1', url: '/v1/offers/'}, {version: 'v2', url: '/v2/offers/'}], function() {
    describe("Suite for version {version}", function() {
      before(function() {
        console.log("I run before the suite of each data-set!");
      });

      it("Simple test for version {version}", function(data, done) {
        data.version.must.include('v');
      });
    });
  });
});

from data-driven.

It would be good if you could provide an example of this. I have tried with the following test code:

var data_driven = require('data-driven')

describe('component under test', function(){

    beforeEach(function() {
        console.log('before test')
    })

    data_driven([{value: 0},{value: 5},{value: -2}], function() {
        it('should do something', function(ctx){
            console.log('in test function with ' + ctx.value)
        })
    })
})

On running, this produces the following output:

  before test
in test function with 0
․before test
in test function with 5
․before test
in test function with -2

So, 'before' is running before each test function, but the data_driven function itself will be called before the beforeEach.

If you need a function to actually generate the test data to pass into data_driven, then you could always just create a function for it e.g.

data_driven(withTestData(), function() {
 it('should test something') {
   ...
 }
}

from data-driven.

My use case requires that the data come from an external yaml source. I had assumed I'd need to do this asynchronously. But it looks like for this case I can just put the yaml data retrieval in a method which is called where you have shown above. I'm still not 100% clear on when async is necessary and when it's not. But this issue (if it is indeed an issue) is no longer blocking my usage of the module.

Thanks again.

from data-driven.

Hi. Sorry for the delay here. I had to set down the data driven testing for a while. But, picking it back up, I'm still looking for support for the following use case:

1. in before method, do various setups which may be running asynchronously, which will result in a set of variable test data
2. based on the test data created in step 1, iterate over a set of mocha tests N times, where N is the number of data sets created in step 1

Here is an example:
/*global describe:true, it:true, before:true, after:true */
"use strict";

var assert = require('assert'),
    seleniumSetup = require('../utils/setupWrapper'),
    dd = require('data-driven'),
    users = [],
    setup;

describe('Setup thingy works @medelman', function() {

    before(function(done) {
        (new seleniumSetup()).setup({
            "view": ['login', 'serviceError'],
            "locator": ["wallet"],
            "user": {
                "nobank": {
                    "type": "personal",
                    "country": "US",
                    "addCreditCard": {
                        "type": "Visa",
                        "confirm": "",
                        "primary": ""
                    }
                },
                "nocc": {
                    "type": "personal",
                    "country": "US",
                    "addBankAccount": {
                        "type": "checking",
                        "confirm": "",
                        "primary": ""
                    }
                }
            }
        }).
        then(function(result) {
            setup = result;

            //setup the data array to drive the test
            users = [];
            for (var i in setup.user) {
                if (i !== undefined && setup.user[i] !== undefined) {
                    users.push(setup.user[i]);
                }
            }
            done();
        })
    });

    after(function(done) {
        setup.driver.quit().then(function() {
            done();
        });
    });
    dd(users, function(done) {
        it('should login with each user account', function(user, done) {
            setup.driver.get(setup.targetBaseUrl);
            setup.view.login.login(user.Email, "11111111");
            setup.view.login.logout().
            then(function() {
                done();
            });
        });
    });


});

in the seleniumSetup.setup method, I pass in some JSON to describe user accounts I want to create and then use in my test below. the setup method accesses services via ssh and shell in order to create those accounts. Before calling "done" in the before method, I put those created users into the "users" array. Then I'd want to run the test "users".length times.

While there might be a way to hack around this by somehow wrapping my "before" functionality in one function which is called in the datadriven function argument and eventually returns an array, that seems to go against the spirit of the "before" architecture. Additionally this makes usage of the setup data elsewhere more problematic.

I hope this provides enough detail to give an understanding of the challenge I'm facing.

from data-driven.

Thanks for the extra information. That certainly helps me understand what you are trying to achieve.

I understand that some before steps do need to be performed asynchronously, however I would ask why the users data cannot be defined / calculated synchronously, and would therefore be available when defining the data-driven tests themselves. This data could then be used by the async before steps if required.

So, you would have your users array representing the users you want to create, and then this would be used in the data-driven section to drive your tests, and then in your before function to actually create users based on that JSON. From the code you have posted, it looks like this would be a relatively simple refactor.

At the moment, there is no clear way for the actual tests to be defined asynchronously, which is effectively what the alternative would be. I'm not convinced that this would even be sensible, as I could imagine it creating issues where on some test runs with the same code, different tests could end up being produced. The tests would be more predictable if they were operating on stable input data.

Does this make sense?

from data-driven.

Hi, I'm not completely sure, this is the exact same thing, but I would also need the before() and after() functions to run before/after each set of tests respectively. In my current use case we're testing an API and the corresponding data in a database. Before a test run, I clear the database, and many of my assertions are based on the fact that the database is empty and therefore e.g. counters start from 0. Now when I run the tests with each set of data, I'd want my database to be cleared before each test run.

Right now the before() and after() methods are only run before or after ALL tests, so for each variant, are run, therefore the assertions fail, because on the "second" run with the second set of data values, the database isn't empty.

Is that understandable? Otherwise I could provide a simplified example...

Thanks a lot.

from data-driven.

After thinking about it, there's three places one could need the before()/after() functions when using data-driven, but right now there's only two implemented.

• * before/after the WHOLE suite
• * beforeEach/afterEach for each single test case
• ! before/after each 'data-driven data set suite'

It would probably be best to have a separate suite for each value, which in turn could have a separate before/after function.

from data-driven.

That's great, thanks for the extra info on before behaviour.

from data-driven.