spongepowered / mctester Goto Github PK

Support for running McTester out of SpongeForge is mostly there - in fact, this repository tests itself via ExceptionTest. A few changes need to be made before it's fully complete:

• Properly clean out the 'mods' directory when moving from a non-Sponge runner (which requires us to download the latest Sponge jar) to a Sponge runner (where we use the Sponge sources that are already on the classpath). Right now, we end up with a leftover spongeforge jar when running under Sponge.

• Document and parameterize the sponge downloader. We currently always download the latest SpongeForge 'stable' build for 1.12.2, but plugins may want to customize this (e.g. test against a fixed build or the recommended build).

Launching the tests using gradle no longer works (it used to work 13 days ago).

I copied the debug test output with stacktrace, and from what I can tell it no longer find the shadow field languageManager in the net.minecraft.client.Minecraft class. I guess something has changed during those last 13 days ?

Here is the gist of the gradle debug :
https://gist.github.com/zessirb/416bf8840638d1ab729ac4b56e97da0e

And here is the build.gradle I'm using (it didn't change and was working before) :
https://github.com/OnapleRPG/Harvester/blob/master/build.gradle

It's not always necessary to spawn a full client. McTester should allow specifying the platform that a test should be run on.

There are two ways of handling a mix of server-only and client tests:

1. Start both the client and server on JUnit startup. This decreases execution time, but dramatically increases memory usage, since both a sever and client are running at the same time.
2. Run all of the tests for one platform, stop the server (or client), then start the platform for the other tests. This decreases peak memory usage, but will probably add anywhere from 30-60 seconds to overall execution time.

There could be a JVM argument to toggle between these modes, since they shouldn't have any impact on test results.

Given three methods working on local (tested on Windows 10 & Linux Mint 18.2), when I try those with travis-ci.org, then one fails without an exception.

This is the file where the method is called ; its name is testMineAllowedBlock.
The method should place a dirt block, mine it with a diamond shovel and check that the block at the given position is not a block of dirt, but it is.

Here are the full debug logs (see 16:27:15.282 for the failing method).

I tried adding some delay in case the travis machine didn't have time to execute every commands in time.

Currently, the Client API deliberately only allows for running one-off commands or actions on the client. There is no Client#runOnClient(Runnable runnable) method that would allow for running arbitrary code on the client thread.

Adding such a method would have the immediate benefit of making tests simpler to write. More complex tests could be written without the need to complicate the Client interface.

The downside to adding such a method is that it would permanently rule out support for out-of-process clients. as described in #1. There is no way to run an arbitrary Runnable on another process, so any multi-client support would have to be accomplished through the multi-classloader approach.

McTester ended up uncovering quite a few race conditions in Minecraft, which I've had to fix.
At a minimum, these issues should be clearly documented, with an explanation of the problem and, ideally, instructions to reproduce it.

It might also be useful to open issues on the Minecraft issue tracker.

I really like being able to write tests in Kotlin using coroutines. However, supporting both normal and coroutine tests imposes a non-trivial maintenance burden:

Specifically, any blocking method - which includes every single method in Client - needs to be duplicated. I don't see any way of getting around this - normal Java code can't call a suspend function, and Kotlin code has to call a suspend function if it wants to avoid blocking.

I see two ways forward:

• Continue to support both Kotlin and Java. We'll have to try and reduce the confusion over which methods to use, however.
• Drop Java support, and only support writing tests in Kotlin. While this would simplify the API and implementation, it would impose a non-trivial barrier for users looking to get started with McTester. For people with no experience with Kotlin, they would need to learn (parts of) a new language to even write a simple test.

From what I've seen, there doesn't seem to be any way of making a single suspendable method callable from both Java and kotlin - and to make it block when called from Java, and suspend when called from Kotlin. If anyone knows of a way to accomplish this, it would be incredibly helpful.

This log has some weird client-thread NPE: https://paste.feed-the-beast.com/view/32261a03

Build: https://travis-ci.org/SpongePowered/McTester/jobs/449533156

This is currently blocked on the following Mixin issues (and possibly other as-of-yet-undiscovered ones):

• SpongePowered/Mixin#276
• SpongePowered/Mixin#277

While a singleplayer environment is usually sufficient for testing purposes, it can sometimes be useful to control multiple clients from a test class. For example, testing SpongeAPI's Keys.VANISH on a player would require at least two clients to be connected.

There are two different ways that we could go about accomplishing this:

• Run each client in a separate 'base' classloader. Normally, LaunchClassLoader is created from a single base AppClassLoader. In this scenario, each client instance would have its own base (non-LaunchClassLoader) classloader. Each base classloader would be responsible for creating its own LaunchClassLoader.
• Run each client as a separate process. In this scenario, there would be no direct communication between the root process's main thread (which runs the JUnit test cases), and the individual client instances. Instead, all communication would continue to occur through the pre-existing custom packet system.

The first approach might make debugging slightly easier - there's only one JVM to attach breakpoints to. On the other hand, the classloader setup could very quickly spiral out of control - any leakage could be a nightmare to debug when there are four active threads (JUnit runner + server + 2 clients) at any given point in time.

The second approach might simplify the implementation - each process can simply be launched with the proper command-line arguments specifying the server to connect to. Once connected, our internal custom packet control system will continue to work as normal. Ideally, there would be no observable difference between the in-process singleplayer client, and the separate-process clients connecting to the LAN world.

Currently, McTester crashes if a plugin adds SpongeVanilla to the classpath (e.g. via a Gradle dependency). We should either support having a dependency on SpongeVanilla, or print a nice error message informing the user that using SpongeVanilla is unsupported.