drogue-iot / drogue-cloud-testing Goto Github PK

Currently we create some resources, but have issues cleaning them up properly. This is due to different idea of different creates we use for testing.

Test dependencies are:

• rstest – For parametrized tests
• test-context – For managing test state

However, rstest and test-context conflict. Also see: la10736/rstest#94

Resources are:

• WebDriver instance
• Applications and devices created using drg
• Applications and devices created using the API

Some of the resources are managed async, and need to be cleaned up async. Some of them are managed sync.

Cleaning up via Drop would be ok, but Drop is tricky to use with async. But this is how we currently clean up the WebDriver, to ensure that it is freed.

One way to deal with this could be to create some kind of Resource<T> wrapper, which you can explicitly drop/destroy. But which is also registered with the test context, to schedule cleanup for the end of the test. When using the test-context crate, this could automatically be cleaned up (async). However, when using rstest based tests, this doesn't work.

Ideally, we would find a way to combine rstest and test-context.

We don't really use that anymore. So drop it, and reduce the build time by a few seconds.

Also run the tests using Minikube.

For this we need nested virtualization, which currently doesn't seem to be supported by most cloud providers.

Websockets

• stream data for an app works
• with bearer token
• with api key
• with bearer token passed as a query parameter
• stream data without read permission fails
• stream data as anonymous permission fails
• stream data as anonymous for a public app works

App administration (console)

• Transfer an app works
• Decline an app transfer works (owner and new owner)
• Cancel an app transfer as another user fails
• Add members to app
• Add members that does not exist fail
• Add members as non owner fails

HTTP Endpoint:

• X.509 client certificates

MQTT Endpoint:

• X.509 client certificates

Currently there seems to exist no functionality to get the test name that is currently being processed. This however should be used to prefix things like filenames of screenshots.

One proposal I saw is to create a test wrapper (using a macro) which injects this into the test being run. This could be done using "thread local", to somehow inject this into the TestContext that we have anyway. So it would be available throughout the test.

I guess the fact that we are using async tests might make this a bit more complicated though.