Giter Club home page Giter Club logo

cg-brutaltester's Introduction

cg-brutaltester

cg-brutaltester is a Java local tool to simulate the CodinGame multiplayer arena. How does it work? Very simple:

java -jar cg-brutaltester.jar -r "java -jar cg-referee-ghost-in-the-cell.jar" -p1 "./myCode.exe" -p2 "php myCode.php" -t 2 -n 100 -l "./logs/"

At the end of the command, you will get something like this:

13:19:47,629 INFO  [com.magusgeek.brutaltester.Main] *** End of games ***
+----------+----------+----------+
| Results  | Player 1 | Player 2 |
+----------+----------+----------+
| Player 1 |          | 7,00%    |
+----------+----------+----------+
| Player 2 | 52,70%   |          |
+----------+----------+----------+

How to read it: Player 1 won 7.00% of the time against Player 2. Player 2 won 52.70% of the time against Player 1. The total is not 100% because you have some draws.

How to build from sources

  1. Install Java 1.8 (JDK)
  2. Install Maven.
  3. Run in command line <path_to_maven>/mvn package inside root directory of this repo.
  4. ./target/cg-brutaltester-0.0.1-SNAPSHOT.jar โ€” is compiled brutaltester! You can rename it to cg-brutaltester.jar to make command line above work.

Now you should get (or compile from sources) referee for specific game and make it work together with brutaltester as stated above.

Command line arguments:

Referee -r <string> (Mandatory)

This is the command line to start the referee process. The referee must respect the cg-brutaltester protocol. See How do I make my own referee? for more information. In our example, we use a runnable Jar file as the referee.

Player X -pX <string> (Mandatory)

Each -pX argument is the command line to start a player process. You can give a maximum of 4 players. But don't forget the some referees will ignore some players (for example, Ghost in the Cell only uses 2 players). In the example, the first player is a simple executable file and the second player is a php file.

Threads -t <int> (Optional; Default is 1)

The number of threads to spawn for the games. If you give 2, it means that you will have 2 games playing at the same time. It's useless to spawn too many threads. If you have a 4-core CPU, you should not try to spawn more than 3 threads.

Number of games -n <int> (Optional; Default is 1)

The number of games to play. The given example will play 100 games.

Logs directory -l <string> (Optional)

You may need the logs of the file. If you specify a directory, all games will be saved in the given directory. The files contain standard and error outputs of all processes (referee and players).

Swap player positions -s (Optional)

There are some games (such as Tron), where one player has a disadvantage from the beginning on because of an asymmetric map. In this case you can repeat the game on the same map, but with positions changed. For more than two players this will perform a simple rotation and not test all permutations (resulting in 4 matches on the same map for 4 players instead of 24). NOTE: not all referees support this flag, as they have to allow setting a seed.

Initial Seed -i <int> (Optional)

It allows to use the same seeds in different runs. You can't select individual seeds, but only the starting seed for the Random Number Generator. It's useful to have repeteable tests.

Old mode -o (Optional)

Since Botters of the Galaxy and Ultimate Tic Tac Toe, Codingame changed a lot the way of creating a referee. Because of that, all games created before Botters of the Galaxy and Ultimate Tic Tac Toe use the "old way". If you want to use an old referee, you have to use this flag.

Verbose -v

Activate the verbose mode. Spam incoming.

Help -h

Display this help :

usage: -r <referee command line> -p1 <player1 command line> -p2 <player2 command line> -p3 <player3 command line> -p4 <player4 command line> [-v -n <games> -t <thread>]
 -h          Print the help
 -l <arg>    A directory for games logs
 -n <arg>    Number of games to play. Default 1.
 -p1 <arg>   Required. Player 1 command line.
 -p2 <arg>   Required. Player 2 command line.
 -p3 <arg>   Player 3 command line.
 -p4 <arg>   Player 4 command line.
 -r <arg>    Required. Referee command line.
 -s          Swap player positions
 -i <arg>    Initial Seed. For repeteable tests.
 -t <arg>    Number of thread to spawn for the games. Default 1.
 -v          Verbose mode. Spam incoming.

How do I make my own referee?

WARNING !

Since Botters of the Galaxy and Ultimate Tic Tac Toe, CodinGame change a lot the way of creating a referee. Because of that, cg-brutaltester had to adapt.

If you want to use or create a referee for a game created before Botters of the Galaxy or Ultimate Tic Tac Toe, use this wiki page.

Your referee must be runnable with a command line (or you won't be able to give it to cg-brutaltester) and you have to use the standard input and output streams. The referee can output on the error stream for debug purposes or real errors. It will be stored in the log file of the game. cg-brutaltester is a very naive arena, and the referee must tell it how to work.

All steps

As an example, you can check my Ultime Tic Tac Toe brutaltester referee To create your own referee for cg-brutaltester, there's X steps to follow:

  • Fork the referee repository
  • Modify the pom.xml file.
  • Add the CommandLineInterface class in the package com.codingame.gameengine.runner

I'm currently thinking of a way to automate the process. Copy/pasting CommandLineInterface is good enough for now, but this is clearly not the best thing to do.

Incoming features

This is not an official roadmap at all.

  • Generate an html file for the results (with graphics!)
  • Better handling of crashing players' code
  • Handle timeouts
  • Conquer the world

List of compatible referees

If you have a bug or a problem with one of these referees, create an issue of the github project of the referee, not on cg-brutaltester project. This may not be a full list of available referees for cg-brutaltester. If you want to add a referee to this list, just make a pull request.

cg-brutaltester's People

Contributors

aangairbender avatar aexg avatar akarachudra avatar bastien-35 avatar coac avatar counterbalance avatar davidb avatar dependabot[bot] avatar dreignier avatar eulerschezahl avatar fala13 avatar jfbogusz avatar johnpage-agixis avatar kevinsandow avatar lpenz avatar panchishin avatar visualdev-fr avatar xoposhiy avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

cg-brutaltester's Issues

problem running the program

I have a problem with cg-brutaltester. I have built the project both locally and using docker. But when I try to run it I get this error

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/logging/log4j/util/ReflectionUtil
        at org.apache.logging.log4j.jcl.LogAdapter.getContext(LogAdapter.java:39)
        at org.apache.logging.log4j.spi.AbstractLoggerAdapter.getLogger(AbstractLoggerAdapter.java:48)
        at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:40)
        at org.apache.logging.log4j.jcl.LogFactoryImpl.getInstance(LogFactoryImpl.java:55)
        at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:655)
        at com.magusgeek.brutaltester.Main.<clinit>(Main.java:16)
Caused by: java.lang.ClassNotFoundException: org.apache.logging.log4j.util.ReflectionUtil
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 6 more

-i and -s only works with -o

Because of the new way of making referees, at the moment -i and -s (initial seed and swap) won't works if you are not using an old referee.

Seed argument also trigger players swap

Hello, weird double number of games when setting the initial seed.

Steps to reproduce:

  • build current brutaltester
  • build cg_fall_2002 referee linked in the readme
  • This runs 1 game: java -jar brutal.jar -r "java -jar -Dleague.level=2 cg_fall_2022.jar" -p1 "python3 do_nothing_bot.py" -p2 "python3 do_nothing_bot.py" -n 1 -t 1
  • This runs 2 games: java -jar brutal.jar -r "java -jar -Dleague.level=2 cg_fall_2022.jar" -p1 "python3 do_nothing_bot.py" -p2 "python3 do_nothing_bot.py" -n 1 -t 1 -i 0

The do_nothing_bot.py:

width, height = [int(i) for i in input().split()]

while True:
    my_matter, opp_matter = [int(i) for i in input().split()]
    for row in range(height):
        for col in range(width):
            (
                scrap_amount,
                owner,
                units,
                recycler,
                can_build,
                can_spawn,
                in_range_of_recycler,
            ) = [int(k) for k in input().split()]

    print("WAIT")

Adding -i 0 to set the seed for consistent testing also double the number of games as if the -s arg was present. This double the testing time for no reason. Failed games, for a timeout, show a command to reproduce the failed game that contains -s in it (don't know if that helps).

Cheers

MeanMax

Hi, two questions:

  • will the MeanMax referee be available when the contest is opened?
  • if yes, will it work with this utility directly?

Thanks

CLI class is outdated

The GameRunner class is now abstract, replaced by MultiplayerGameRunner. I updated it here if you want to see. I disabled the refereeInput part ('-d') option. The MultiplayerGameRunner has a setSeed() method that should be enough in most cases, if needed.

Botters of the Galaxy

Are there plans to build a version of the referee of Botters of the Galaxy compatible with the brutal tester?

Bunch of errors when testing with SpringChallenge 2021 ref (java 16.0.1 on windows 10)

I tried a couple of different things but didn't put my hands into the code (yet). I took the released jars on github and also tried with a local compiled version. Same thing.
I just hope someone knows what this is about by just looking at those lines (I'm not familiar with java).
Thx.

22:33:21,063 ERROR [com.magusgeek.brutaltester.GameThread] Problem with referee output in game1. Output content:WARNING: sun.reflect.Reflection.getCallerClass is not supported. This will impact performance.
null
null

22:33:21,064 ERROR [com.magusgeek.brutaltester.GameThread] Error during game 1
22:33:21,065 ERROR [com.magusgeek.brutaltester.GameThread] Exception in thread "Thread-0" java.lang.ExceptionInInitializerError
at com.google.inject.internal.cglib.reflect.$FastClassEmitter.(FastClassEmitter.java:67)
at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
at com.google.inject.internal.cglib.core.$DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
at com.google.inject.internal.cglib.core.$AbstractClassGenerator.create(AbstractClassGenerator.java:216)
at com.google.inject.internal.cglib.reflect.$FastClass$Generator.create(FastClass.java:64)
at com.google.inject.internal.BytecodeGen.newFastClass(BytecodeGen.java:204)
at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.(ProviderMethod.java:256)
at com.google.inject.internal.ProviderMethod.create(ProviderMethod.java:71)
at com.google.inject.internal.ProviderMethodsModule.createProviderMethod(ProviderMethodsModule.java:275)
at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:144)
at com.google.inject.internal.ProviderMethodsModule.configure(ProviderMethodsModule.java:123)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:349)
at com.google.inject.spi.Elements.getElements(Elements.java:110)
at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
at com.google.inject.Guice.createInjector(Guice.java:96)
at com.google.inject.Guice.createInjector(Guice.java:73)
at com.google.inject.Guice.createInjector(Guice.java:62)
at com.codingame.gameengine.core.RefereeMain.start(RefereeMain.java:62)
at com.codingame.gameengine.runner.RefereeAgent$1.run(RefereeAgent.java:68)
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @60285225
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:357)
at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
at com.google.inject.internal.cglib.core.$ReflectUtils$2.run(ReflectUtils.java:56)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:312)
at com.google.inject.internal.cglib.core.$ReflectUtils.(ReflectUtils.java:46)
... 21 more
java.lang.RuntimeException: Invalid Referee command:
at com.codingame.gameengine.runner.GameRunner.readCommand(GameRunner.java:318)
at com.codingame.gameengine.runner.GameRunner.readGameInfo(GameRunner.java:302)
at com.codingame.gameengine.runner.GameRunner.runAgents(GameRunner.java:127)
at com.codingame.gameengine.runner.GameRunner.runGame(GameRunner.java:453)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:78)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:567)
at com.codingame.game.CommandLineInterface.main(CommandLineInterface.java:68)

Infinite loop ?

Hello, I tried the brutaltester with the spring challenge 2021 referee, I compile my bot made in Rust, launched the command and it's just looping, no information...

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.