Tests are not consistent and some are not skipping some tests about dart HOT 11 CLOSED

These are great observations @Stargator.

It seems that none of the tests are skipped. Should this be changed?

I think probably yes—if we skip tests in any of the exercises, I think we should skip them in all of them. I've written a little bit about the background for why I originally added skips to tests, and why I think it's probably the right thing to do: test suite anatomy

I wasn't sure if there may be an additional parser that would made the change before giving it to a user.

No, the tests are delivered as-is from the repo.

My thinking was perhaps they are not set to true in order to run the tests in this project.

This is always a bit tricky. We want to provide the best possible experience for the end-user, but we also don't want to have to edit the tests locally every time in order to run the tests. We also don't want to accidentally check in the changes if we do edit them to run the tests while working on the exercises. I wrote a bit about this in the tooling and ci doc for the "launch a track guide", in case that helps here.

Additionally, the test files are organized differently from each other. I think we want consistency in the format and to skip all but the first test by default.

Yeah, I think we should be consistent wherever we don't have a compelling reason to do otherwise.

Often it takes a while in a newer track to figure out all the patterns that we want, so it feels like it's the right time to think about this now.

@SuperPaintman You've got the most context about the history and background of this track. I'd love to hear your perspective on all of this, since you might well have made some decisions deliberately/carefully, and we might not see all the reasons.

from dart.

There can be a compelling reason to introduce "groups" into the test files. Cause they are a concept used by the framework to organize tests we may want to slowly expose users to them over the course of exercises.

from dart.

Nice! Yeah, I think we have some exercises where this would be a natural fit.

Does the test suite also allow you to run "focused" tests in any way (by line number or test name or group or whatnot?)

from dart.

By group name, yes. Not sure about line number.

As for test name, I think tests are automatically run once they are defined. This can be seen by the group functionality.

From the Difference of Squares test:

void squareOfSums() { // Wrap the group with a function and then wherever the function is called the group will be called as well.
    group("#squareOfSums()", () {
      test("square of sum 1", () {
        expect(squares.squareOfSums(1), equals(1));
      });

      test("square of sum 5", () {
        expect(squares.squareOfSums(5), equals(225));
      });

      test("square of sum 100", () {
        expect(squares.squareOfSums(100), equals(25502500));
      });
    });
}

Then there are tags, which can be applied to individual tests, groups, or suites. Then either by running an argument with pub run test or configuring dart_test.yaml, you can specify which tests are run by tags.

from dart.

Interesting. So another option we have, instead of skipping, would be to use a tag for the active tests to run, and then people would add the tags. Then you could run the whole test suite by just not specifying any tags.

from dart.

I'm going to suggest we add tags to a later exercise(s) and just use the first 10 exercises to introduce the language, the test framework, the package manager, and analysis tool. especially for beginners that's a lot to take in.

from dart.

Speaking of which, I noticed none of the current tests seem to be dependent on another. Are we not going to guide users into the language/framework/tools by controlling the first 10 exercises they work on?

from dart.

I noticed none of the current tests seem to be dependent on another.

Do you mean that none of the tests for any of the exercises depend on tests for any other exercises?

Are we not going to guide users into the language/framework/tools by controlling the first 10 exercises they work on?

We have a whole lot of control.

We can set "core": true on 8 to 20 exercises, which will need to be completed in the order specified in the exercises array in config.json. All of the core exercises must be completed in order to complete the track. In the normal usage of the v2 site, you cannot move on to the next core exercise until the mentor approves your solution.

Each core exercise should unlock about 3 additional (optional) exercises. You don't have to complete any of these in order to complete the track, however we expect people to do so, while they're discussing their core exercises with mentors.

You will typically get feedback from mentors on optional exercises, as well.

So we can do a lot to guide people along the way, in particular we can add this guidance to the core exercises.

We should not have dependencies between exercises, I think. Each exercise should be stand-alone, though the knowledge learned from previous exercises can be relevant to solving the next one.

from dart.

Do you mean that none of the tests for any of the exercises depend on tests for any other exercises?

I meant to say, none of the exercises block other exercises. @kytrinyx's response to the other comment was helpful.

Then I think we should determine what order we want the current exercises to be in. Then we can determine where to introduce concepts like groups, setup/teardown, etc.

from dart.

Then I think we should determine what order we want the current exercises to be in.

Agreed.

from dart.

@kytrinyx Closing this discussion to focus talks about the order of the exercises in another thread.

from dart.