GDBMeter (Graph Database Metamorphic Tester) is a tool that automatically tests Graph Database Management Systems (GDBMS). It tries to find bugs that cause the GDBMS to fetch an incorrect result set.

Required Software:

• JDK 11
• Gradle 8.0.2 which we provide bundled with our source code
• The graph database you want to test (Neo4J and JanusGraph are both embedded into the JAR for ease of use)
  • We recommend testing RedisGraph via a Docker container

The following commands clone the repository, build a JAR file and start GDBMeter:

./gradlew clean shadowJar
cd build/libs
java -jar gdbmeter-*.jar -db neo4j -o partition -v

When running GDBMeter in verbose mode (-v) the queries that were executed are printed periodically. If GDBMeter does not find any bugs, it executes infinitely. To see all available options one can use the -h option.

Our testing approach, called Predicate Partitioning, is based on the idea that we can partition a result set through a predicate into disjoint subsets. Combining these subsets should again give the complete original result set. This approach has been successfully applied to SQL databases and has been termed Ternary Logic Partitioning (TLP).

To test the RedisGraph database, an external server has to be started. To do this we recommend the following docker command which starts the server in the appropriate version:

docker run -p 6379:6379 -it --rm redis/redis-stack-server:6.2.2-v5

GDBMeter generates logs in the logs folder. Every individual query is logged and for JanusGraph the schema is serialized as part of the log. If GDBMeter finds a bug, the log can be used to reproduce the bug.

To reproduce a bug simply find the last logged message of the form Finished iteration, closing database. And remove everything above it. This leaves only the queries that were run during the latest iteration. Place these in a file called replay in the logs folder. Re-running GDBMeter with the option --replay or -r then runs only those queries that are present in replay. At this point a manual reduction is necessary. Usually, the best way to do this is to remove part of the queries and see if the bug still occurs. Continue this until a minimal example is left. We plan on adding support for an automatic reduction test based on Delta Debugging.

In case you found a bug using GDBMeter we would appreciate it if you mention our project when reporting the bug. We would, of course, be very happy if you use GDBMeter or if you would like to contribute.

GDBMeter also supports other, less successful, testing approaches: