I agree this aspect needs some special attention, thanks for looking into it.

Before all else, we should be clear on the difference between a browser based setup and a platform-agnostic setup.

Platform-agnostic (client and server)

Pros

• Early feedback from unit tests on whether your app will be SSR-compatible (eliminates headaches that plague au1 projects)
• Even without SSR, unit tests running in node mode start up faster, require fewer tools and are much easier to debug in vs code
• Fewer surprises in the planning of a project / risks easier to manage

Cons

• More boilerplate
• Higher learning curve

I think for many people, even those who don't necessarily need SSR, the major long-term benefits of the pros likely outweigh the relatively minor initial drawbacks of the cons.

How it works

In very simplified form, the idea is this:
setup-browser.js

import { TestContext } from '@aurelia/testing';

TestContext.create = function () {
  return { document };
};

require('**/*.spec.js');

setup-node.js

import { TestContext } from '@aurelia/testing';
import { JSDOM } from 'jsdom';

TestContext.create = function () {
  const { document } = new JSDOM();
  return { document };
};

require('**/*.spec.js');

some-spec.js

import { TestContext } from '@aurelia/testing';

it('works', function () { 
  const { document } = TestContext.create();
  // ...
});

The document is an always-available global in the browser, but has to be retrieved from an instantiated JSDOM object in SSR. @aurelia/testing exposes a TestContext module-scoped variable as a convenience api that wraps these differences in setup, allowing you to reuse the exact same unit tests in the browser as well as in nodejs.

The ones used by the framework are: setup-browser and setup-node.
More stuff happens in there, and there is more convenience api, but the above is the general idea.

To run the tests in nodejs, you simply run mocha in native node and point it to the setup-node file, and to run them in the browser you run e.g. karma and point it to the setup-browser file. This gives you two complete test suites for the price of one.

Browser-only

Pros

• No specific bootstrapping needed; more api freedom
• No specific need for a single-entry test setup file per environment; tests can be run per individual file for example.
• Simpler and easier to understand overall

Cons

• Tests only work in the browser (and maybe in jest)
• Promotes ways of programming that are incompatible with SSR (at least for now) / no easy protection against breaking SSR

Browser-only is probably preferable for most types of internal-facing websites (no need for SEO or specific startup perf metrics, etc) and demo's/pocs (due to the simplicity, and ease of getting started).

Start/stop

A simple solution, just like we do in most integration tests, where setup calls start and tearDown calls stop:

const { app, host, tearDown } = await setup('<my-app></my-app>', MyApp);

expect(div.textContent.trim()).to.equal('Hello Aurelia 2!');

await tearDown();

Conclusion

I'm not sure whether putting a browser-specific setup in @aurelia/testing is a good idea because the general spirit of all aurelia repo packages is to be platform-agnostic.

And, given how simple of a method it is, I feel like showing this snippet in the docs (and maybe even including it in a generated project boilerplate in a util file) could be beneficial in itself, as it lays bare exactly how it works and invites people to tinker with it and make it suit their needs. If we put it in a package, we'll inevitably get feature requests to put more stuff in it and turn it into something more complex, which kind of beats the purpose at that point.

I'm not a particularly big fan of the au1 testing api, I think it tries to do too much and yet is capable of too little at the same time. But that's just my opinion.

from new.

.delegate in Aurelia attaches an event listener to nearest dom boundary, most of the time it will be the document, which means even if we don't do anything that could potentially leak in our code, an aurelia instance should always call .stop() in testing environment, to avoid unexpected call, and mem leak

from new.

The nodejs tests currently are not available for user app in this repo.

Have to wait for jest loader for au2 conventions, which will be done after au2 styles refactoring.

Our @aurelia/testing should provided some setup() helper for platform-agnostic setup. Should we transfer this issue to aurelia/aurelia repo?

from new.

How does this look?
https://gist.dumber.app/?gist=79b0cbde9118a15c467357c381bf9201&open=test%2Fsetup.ts&open=test%2Fapp.spec.ts

from new.

It feels to me, the setup-browser is too verbose to end users.
If we don't want to put it in @aurelia/testing, we should provide it in @aurelia/testing-html (for browser env and jest fake browser env) and @aurelia/testing-nodejs (for nodejs env, dep on jsdom).

from new.

May I ask can/should user skip TestContext and TestConfiguration in unit tests? Just like our existing template (and also my first demo in this issue) in the repo?

from new.

@bigopon stop can also be called by the helper if I add a callback instead of pretty async/await.
https://gist.dumber.app/?gist=2e3c025b325b87bbbae344d758ae269b&open=test%2Fhelper.js&open=test%2Fapp.spec.js

from new.

@bigopon

.delegate in Aurelia attaches an event listener to nearest dom boundary, most of the time it will be the document, which means even if we don't do anything that could potentially leak in our code, an aurelia instance should always call .stop() in testing environment, to avoid unexpected call, and mem leak

Delegate doesn't work in detached dom anyway. I would say in detached dom, calling stop is not a hard requirement because as @3cp eluded the node is the GC root.

@3cp

The nodejs tests currently are not available for user app in this repo.

I'm just pointing out that it's important to be aware of this necessity for some people. In principle I don't have anything against a minimalistic browser-specific setup, just keep in mind that we need to have an agnostic one eventually as well, if that should impact the design somehow.

Have to wait for jest loader for au2 conventions, which will be done after au2 styles refactoring.

Jest is intended to make node pretend like it's a browser, so a browser-specific setup should work as-is in node anyway. That's not what I meant with running tests in node. I mean running them in regular node, so no jest tampering with globals. For example in mocha, like we do in the repo.

Our @aurelia/testing should provided some setup() helper for platform-agnostic setup. Should we transfer this issue to aurelia/aurelia repo?

Sure. Don't forget we also have this issue: aurelia/aurelia#509

@3cp

How does this look?
https://gist.dumber.app/?gist=79b0cbde9118a15c467357c381bf9201&open=test%2Fsetup.ts&open=test%2Fapp.spec.ts

That would be the proper browser variant of platform-agnostic tests.

@3cp

It feels to me, the setup-browser is too verbose to end users.
If we don't want to put it in @aurelia/testing, we should provide it in @aurelia/testing-html (for browser env and jest fake browser env) and @aurelia/testing-nodejs (for nodejs env, dep on jsdom)

Maybe, but I also kind of consider it like part of the config. There are plenty of project boilerplates out there, even in the demo, where a fairly bulky file is essentially part of the user code. Users normally don't do much with it or even look at the file, but they easily can if they need to.

Putting it in separate packages works, but that in turn means more stuff to install and it's already a bit more unwieldy than I'd like it to be. Not a huge fan of small wrapper packages tbh.

@3cp

May I ask can/should user skip TestContext and TestConfiguration in unit tests? Just like our existing template (and also my first demo in this issue) in the repo?

These api's are just convenience wrappers to reduce test boilerplate code. Quick easy access to the lifecycle, scheduler, container, composition root, app, host node, etc. Stuff that you very often use in many real world tests. I would strongly suggest to look at some of the more complicated integration tests in the aurelia repo to see what the possible value is of this api. It's your judgement call in the end.

@3cp

@bigopon stop can also be called by the helper if I add a callback instead of pretty async/await.
https://gist.dumber.app/?gist=2e3c025b325b87bbbae344d758ae269b&open=test%2Fhelper.js&open=test%2Fapp.spec.js

We've already been trial-and-erroring through something similar when @Sayan751 was working on i18n. Here's a nicely working example of an it wrapper that takes care of setup and teardown: https://github.com/aurelia/aurelia/blob/master/packages/__tests__/integration/integration.spec.ts#L51

from new.

Great! That cleared my doubts.

I mean running them in regular node

I just want to clarify an au2 conventions loader (require.extensions) is required for regular node too. Otherwise, user has to write explicit @customElement decorator.

from new.

I just want to clarify an au2 conventions loader (require.extensions) is required for regular node too.

That will most likely be AOT / runtime-node assisted; the same machinery that SSR would be based on. So that's where we'll probably not use an external bundler initially, and instead directly integrate your convention code with AOT

from new.