[Feature Request] Better Documentation for Unit Testing about vue-test-utils HOT 13 CLOSED

@eddyerburgh I'm now leaning towards changing it to a general Testing page with a Unit Testing section, which would provide:

• a couple simple examples of how to test Vue components without vue-test-utils
• an example of where unit tests can get trickier (as a segue into vue-test-utils)
• a brief description of the problems vue-test-utils solves, with a link to a getting started guide in vue-test-utils' own docs

What do you think? And since you're literally writing the book on testing Vue applications, 🙂 would you like to author that page - or at least the unit testing section?

from vue-test-utils.

I agree with @eddyerburgh that we can't provide users a simple answer on what to test. I also find that what to test in Vue isn't really different from other applications. However, since I sadly don't know of a single resource that provides a really good answer to this question (though I'm sure one must exist somewhere), I think we can provide some guidelines.

Personally, I like to prioritize tests by asking 2 categories of questions:

• Importance: How bad would it be if it broke?
• Urgency: How likely is it to break?

Let's apply these to a hypothetical component wrapper for font-awesome and other icons:

• Importance: If it broke and icons weren't rendering anymore, it would decrease the quality of the experience and hurt the brand image. For a chain of restaurants, this would probably just be mildly annoying. For an online store, where trust needs to be high, it could permanently alienate customers.
• Urgency: If it's a really simple component that is not frequently changing, it's probably extremely unlikely that it will break. If we're adding new icons all the time and making changes to the interface of the component, there's more risk.

Then I use these answers to generally prioritize tests in this order:

1. When important and urgent, always include robust tests before shipping, erring towards over-testing.
2. When important, but not urgent, write the 20% of tests that will catch 80% of bugs before shipping, but catch the last 20% of bugs before any significant refactors that touch this code.
3. When urgent, but not important, write the 20% of tests that will catch 80% of bugs, ideally before shipping, then only add more if something breaks.
4. When not important and not urgent, don't test.

This is a slight over-simplification because importance and urgency aren't binary, but on a continuum. Their evaluation can also be pretty complex and answers are inevitably subjective. That means these are tools to be used in conversations, not a simple rubric that can be mindlessly followed.

The team and development process for each project also have to be considered. For example, if the plan is to eventually hand things off to a different team, tests serve a larger role. If there's a strict deadline, so developers have to move fast, it will be more important not to over-test code that is frequently changing. Usually, there are a lot of conflicting priorities and we don't always have all the information we'd like.

These tools for prioritization can also be used at the micro level: when you know you want to test a feature, figuring out which parts/levels to test. I've seen a lot of tests to check that basic Vue functionality works (e.g. the starting value of a data property is set to the correct value). This is an example of testing something of extremely low urgency, since they very rarely catch any bugs.

I think more often, they lead to developers not wanting to touch code, for fear of having to hunt down and fix breaking tests when nothing's actually broken. Broken tests become just a sign that "we changed some code today." This leads to the opposite of the intended effect, where devs see broken tests during CI and think, "Oh, it's probably nothing. We want to get this feature out, so let's ship to prod and we'll fix the tests later." Inevitably, it turns out something really was broken - you just didn't trust your tests, because they're usually lying. (To be fair, I think the docs I wrote are to some extent to blame for this pattern, since we currently use only very simple and contrived examples.)

@blocka Is that the kind of information you're looking for?

from vue-test-utils.

from vue-test-utils.

We've done a huge overhaul of the docs. There's lots of info on what to test, test runners to use and how to set up Vue test utils:

https://vue-test-utils.vuejs.org/en/

Closing this issue, but happy to reopen if you think anything is missing from the docs 🙂

from vue-test-utils.

@willypt You're right, we need better documentation.

I think @chrisvfritz will be updating the docs once vue-test-utils is released.

There's a guide section in vue-test-utils. We will add examples there in the future that might help define the right way to unit test Vue.

from vue-test-utils.

Haha, yes I'll write the page 😛

from vue-test-utils.

@blocka I test all logic inside my components. Often that means using selectors to assert the output. I understand that it can seem brittle. You could use general selectors to avoid having to rewrite tests if your designers decide to rename classes - but tests on the VDOM are always going to be tightly coupled to the markup.

Apart from that you can test component methods (although if you aren't asserting the VDOM you can do this without mounting the component). But most of my tests assert the VDOM/ DOM.

from vue-test-utils.

from vue-test-utils.

@blocka Yep I totally understand your concern. Testing UI components can be tricky, and as you said - there is no golden rule.

I'd be interested to see what solutions you come up with for your project at the moment 😀

And agreed, snapshot testing is definitely useful. I'm going to look into that once vue-test-utils is released. Currently we're having problems with shallow and snapshot testing in Jest.

from vue-test-utils.

from vue-test-utils.

@blocka do you mean that you wrap the already simple button into a component so that you can test it? Yes it is overkill imo 🤣

from vue-test-utils.

Sounds good. Great stuff to put in the manual 😄
Right now we're writing our first app with actual tests. We've written scores of apps with vue, and many more in pre-vue days, but never with tests. We know we've been burned in the past by code that accidentally broke something else we weren't thinking of, and so we know we definitely need some kind of tests.

At the moment I'm settling on a strategy of, "let's see how long I can get a way with seeing If I can interpret, in code, with what I would would testing manually by looking at my browser. Practically speaking this means writing classical unit tests (especially when testing state management stuff), and "light" integration tests (where a mocked system is set up, but there is no specific SUT...we are testing a specific "story" (maybe these are acceptance tests?)

from vue-test-utils.

https://vue-test-utils.vuejs.org/ is updated link i think . /en/ not working in my side.

from vue-test-utils.