Giter Club home page Giter Club logo

debs-gc-2018's Introduction

DEBS Grand Challenge 2018

Here you can find some helpful sourcecodes which should ease your integration into the HOBBIT platform and participation in the DEBS Grand Challenge 2018.

The sourcecodes use the HOBBIT Java SDK and should allow challenge participants to debug their systems using benchmark workload locally without having a running platform instance. The benchmarking system can be tested as pure java code or/and being packed (automatically, by demand) into docker image. Finally the docker image of the system may be uploaded into the HOBBIT online platform (as described here and here) and executed there under the online DEBS GC 2018 benchmark, which will be used for the challenge.

Dataset

The datasets can be requested here.

Benchmark description

The reposity containts the following components:

  • Task generator
  • Sample system
  • Sample system test

The DEBS GC 2018 benchmark sends a stream of tuples to the benchmarking system and expects responces (predictions) from the system to be ranked by defined KPIs. Each next data tuple for each particular ship can be received by the system only if the responce (prediction) for the previous tuple for this ship was sent from the system to the evaluation storage, which acknowdleges data generator. A simplified version of data generator is included to the repository to allow participants to get the stream of training data directly to the sample system. A simplified versions of any other benchmark components (benchmark controller, task generator, evaluatuion storage, evaluation module) required for local debugging will be automatically downloaded as docker images by the docker software.

The online DEBS GC 2018 benchmark benchmark will use another implementations (docker images) of components (data generator, task generator, evaluation storage) where acknowledgement messages are be encrypted.

Usage

Before you start

  1. Make sure that Oracle Java 1.8 (or higher) is installed (check by the java -version). Or install it by the sudo add-apt-repository ppa:webupd8team/java && sudo apt-get update && sudo apt-get install oracle-java8-installer -y.
  2. Make sure that docker (v17 and later) is installed (or install it by sudo curl -sSL https://get.docker.com/ | sh)
  3. Make sure that maven (v3 and later) is installed (or install it by sudo apt-get install maven -y)
  4. Add the 127.0.0.1 rabbit line to /etc/hosts (Linux) or C:\Windows\System32\drivers\etc\hosts (Windows)
  5. Clone this repository (https://github.com/hobbit-project/DEBS-GC-2018.git)
  6. Open the cloned repository in any IDE you like.
  7. Make sure that hobbit-java-sdk dependency (declared in pom.xml) is installed into your local maven repository (or install it by executing the mvn validate command)

How to create a system for existing benchmark

  1. Please find the basic HOBBIT-compatible system implementation in SystemAdapter.java. You may extend it with the logic of your future system. More details about the design of HOBBIT-compatible system adapters can be found here and here.
  2. Put the training data set under the data folder (if required) and modify path to it in DataGenerator.java.
  3. Run the checkHealth() method from the SampleSystemTest.java to test/debug your system as pure java code. Running under sudo may be required, if containers not starting (message such as Got exception while trying to request the creation of an instance of the ...).
  4. Once your system correctly tested as pure java code you may test it being packed into docker container. To build docker image you for the system you have to configure values in the SamplesDockersBuilder.java, package your code into jar file (mvn package -DskipTests=true) and execute the buildImages() from the SampleSystemTest.java. Image building is automatic, but is on-demand, i.e. you have to check the actuality and rebuild images (inc. rebuilding jar file) by your own.
  5. Run the checkHealthDockerized() method from the SampleSystemTest.java to test/debug your system as docker container. All internal logs from containers will be provided. You may skip logs output from other components via skipLogsReadingProperty().
  6. Once you have tested docker image of your system you may upload it into the HOBBIT platform. Please follow the instructions of the standard procedure (decribed here and here) and skip the image building phase.

How to upload your system to the platform

  1. Create a user account via GUI of the platform.
  2. Create a new project at platform's GitLab.
  3. Modify the GIT_REPO_PATH and PROJECT_NAME in Constants.java to fit the URL of project you've just created. SYSTEM_IMAGE_NAME will be used as URI of your system's image (<imageUri>) and should finally look like this git.project-hobbit.eu:4567/<yourUsername>/<yourProjectName>/system-adapter.
  4. Build docker image using the buildImages() from the SampleSystemTest.java. The full URI of build image (<imageUri>) will be shown in console. Don't forget to package/repackage your codes by the mvn package -DskipTests=true before building/rebuilding the image.
  5. Login to the remote gitlab from console: sudo docker login git.project-hobbit.eu:4567 using the credentials of an account you've just created.
  6. Push your image to remote gitlab by the docker push <imageUri>.
  7. Put the modified system.ttl into you project at GitLab. The details about the file see in the section below.
  8. Find your system in the GUI under the Benchmarks Tab after selecting the DEBS GC 2018 Benchmark. The GUI will apply the updated system.ttl during 30-60 seconds after it has been changed.

How to register for the online challenge

  1. Once your system works well in the online platform you may register to the challenges (the training phace and final execution). Please register your system for the both tasks of the challenges.
  2. Systems registered for the DEBS Grand Challenge 2018 Training phase will be executed periodically (till the mid of May) over as some part of unseen data. The results will be publicly available at the online leaderboards. Participation in training phase is not mandatory, but it should help participants to solve all the problems before the final execution.
  3. Systems registered for the DEBS Grand Challenge 2018 Final Execution will be executed once (after the mid of May) over the rest part of unseen data. The results announcement as well as winner award will be held during the DEBS Conference by 25-29 of June.

Benchmark-sensitive information

Please find the example of system.ttl. For your system you have to modify the following:

  • System URI - (in the line <SystemURL> a hobbit:SystemInstance) - some unique identifier of your system (used by the platform internally).
  • Label and Comment - will be displayed in GUI. Please include something (organization name/team name/email/etc...) in the label/comment to identify your team for organizers.
  • ImageName: the URL at which your docker image (<imageUri>) was uploaded/pushed.

BenchmarkAPI (the line hobbit:implementsAPI <http://project-hobbit.eu/sml-benchmark-v2/API>;) should remain unchanged.

In order to switch your system to the query of the current benchmark run (Query 1 or Query 2) the SYSTEM_PARAMETERS_MODEL environment variable will be specified by a benchmark. In case you are not using the provided SampleSystem, you have to implement parsing of the SYSTEM_PARAMETERS_MODEL environment variable in the initialization of your system. The SYSTEM_PARAMETERS_MODEL has the following format: { "@id" : "http://jenaKeyValue.com/URI", "queryType" : "2", "@context" : { "queryType" : { "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType", "@type" : "http://www.w3.org/2001/XMLSchema#int" } } }.

News

1 May 2018: Participants, who submitted working system into the platform please email us with a link to any succesfull run and receive an additional training set.

16 Apr 2018: Sample system implemetaion in Python was announced.

14 Mar 2018: The DEBS GC 2018 Benchmark is available online as well as challenges have been created. Please find the benchmark-sensitive information and helpful instructions above.

9 Mar 2018: The repository and remote docker images were updated. New code shows evaluation of you results predicted by your system. Data Generator was replaced by Task Generator, which still requires training dataset to be downloaded and placed into the data folder. Please delete old-one docker images from your local cache (using docker rmi <imageName>) in order to new images being downloaded by the SDK. The online benchmark will be announced soon.

FAQ

Feel free to ask any questions regading the DEBS Grand Challenge 2018 under the Issues tab.

Please leave a request if you need a python-based example of the sample system.

debs-gc-2018's People

Contributors

smirnp avatar

Watchers

avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar avatar

Forkers

nihlaakram florianschmidt1994 hyungkun

debs-gc-2018's Issues

Interpreting of evaluation log

Hi @smirnp. In the logging, I found some lines like the following

2018-03-20 13:03:38,656 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 44 sent. Curr: 0 tuples/s>
2018-03-20 13:03:39,538 DEBUG [eval-storage-dockerizer] - <2018-03-20 13:03:39,315 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 44 (0 t/s) Actuals: 43 (0 t/s)>

2018-03-20 13:03:39,657 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 44 sent. Curr: 0 tuples/s>
2018-03-20 13:03:40,552 DEBUG [eval-storage-dockerizer] - <2018-03-20 13:03:40,316 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 44 (0 t/s) Actuals: 43 (0 t/s)>

Could you please kindly explain what these number means, especially the <Expectations: 44 (0 t/s) Actuals: 43 (0 t/s)> tuple. Thanks!

Exact format for timestamp in query 2 response

Hello,

Could you please confirm the exact format expected by the benchmark for the timestamp string in the query2 response? Is this similar to the format in the training data, e.g., "24-04-15 10:22". And if yes the 1 digit hours are preceded by a 0 or not - "24-04-15 9:58" or "24-04-15 09:58"? (if I'm not wrong I think that in last training data set they're not).

Thank you,
Emanuel

Meaning of "SHIPTYPE"

Hi guys,
I found in the training data field named "SHIPTYPE" with 27 different integer values.
Can any help me understand what does it mean?
Thanks!

System not showing up on benchmark website

I am using the following system.ttl

@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix ex: <http://example.org/> .
@prefix gerbil: <http://w3id.org/gerbil/vocab#> .
@prefix hobbit: <http://w3id.org/hobbit/vocab#> .
@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix owl: <http://www.w3.org/2002/07/owl#> .

<http://florianschmidt.me/debs2018/debs2018-solution>  a hobbit: SystemInstance ;
    rdfs:label	"DEBS2018TUD"@en;
	rdfs:comment "DEBS2018TUD"@en;
	hobbit:implementsAPI <http://project-hobbit.eu/sml-benchmark-v2/API>;
	hobbit:imageName "git.project-hobbit.eu:4567/florian.schmidt.1994/debs2018solution/system-adapter:latest".

The image was pushed with the following logs

docker push git.project-hobbit.eu:4567/florian.schmidt.1994/debs2018solution/system-adapter:latest
The push refers to repository [git.project-hobbit.eu:4567/florian.schmidt.1994/debs2018solution/system-adapter]
9d3b43b958d3: Pushed
d31910b494f4: Pushed
87b152e7db9f: Pushed
0e9f8a20002d: Pushed
0f2fbc602595: Pushed
59a5ed91aa75: Pushed
43efe85a991c: Pushed
latest: digest: sha256:27746aaf7533d765a948e91390b573afe6e32421144b7a67bd52d2e4376ea4a4 size: 1793

I don't see my solution in the UI of the benchmark (DEBS GC 2018 Benchmark). Is there maybe something wrong with my system.ttl? Do I need to add the latest tag or can that be dropped?

create instance exception with data-generator

Hello,
I get an error when running mvn test. bellow the logs:

2018-04-23 15:36:04,740 DEBUG [Jena] -
2018-04-23 15:36:07,426 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,427 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initializing...>
2018-04-23 15:36:07,550 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-04-23 15:36:07,550 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,551 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initialized. Waiting for termination signal>
2018-04-23 15:36:07,581 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller>
2018-04-23 15:36:07,581 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,581 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,583 DEBUG [benchmark-controller-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-04-23 15:36:07,690 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-04-23 15:36:07,723 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/smirnp/sml-benchmark-v2/system-adapter>
2018-04-23 15:36:07,726 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,766 DEBUG [benchmark-controller-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-04-23 15:36:07,799 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-04-23 15:36:07,799 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-04-23 15:36:07,874 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter] -
2018-04-23 15:36:07,879 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter_0] - <SystemModel: {
"@id" : "http://jenaKeyValue.com/URI",
"queryType" : "1",
"@context" : {
"queryType" : {
"@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
"@type" : "http://www.w3.org/2001/XMLSchema#int"
}
}
}

2018-04-23 15:36:07,880 DEBUG [org.hobbit.sdk.utils.ComponentsExecutor] -
2018-04-23 15:36:07,891 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <SYSTEM_READY_SIGNAL signal received>
2018-04-23 15:36:08,088 DEBUG [benchmark-controller-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=6bd4b08447b3485a4d0bf0fb60d1db734caf5e2e7e53e5b693c8ec73ced85809)>
2018-04-23 15:36:08,528 DEBUG [benchmark-controller-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-04-23 15:36:08,976 DEBUG [benchmark-controller-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=6bd4b08447b3485a4d0bf0fb60d1db734caf5e2e7e53e5b693c8ec73ced85809)>
2018-04-23 15:36:08,976 DEBUG [benchmark-controller-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-04-23 15:36:10,255 DEBUG [benchmark-controller-dockerizer] - <2018-04-23 13:36:11,440 DEBUG [Jena] -
2018-04-23 13:36:12,109 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Init()>
2018-04-23 13:36:12,148 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <BenchmarkModel: {
"@id" : "http://jenaKeyValue.com/experimentUri",
"generatorLimit" : "1000",
"generatorTimeoutMin" : "60",
"queryType" : "1",
"@context" : {
"generatorLimit" : {
"@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorLimit",
"@type" : "http://www.w3.org/2001/XMLSchema#int"
},
"generatorTimeoutMin" : {
"@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorTimeoutMin",
"@type" : "http://www.w3.org/2001/XMLSchema#int"
},
"queryType" : {
"@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
"@type" : "http://www.w3.org/2001/XMLSchema#int"
}
}
}

2018-04-23 13:36:12,149 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Gen.EnvVariables: QUERY_TYPE=1, GENERATOR_LIMIT=1000, GENERATOR_TIMEOUT=60>
2018-04-23 13:36:12,149 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createDataGenerators()>

2018-04-23 15:37:10,941 ERROR [benchmark-controller-dockerizer] - <2018-04-23 13:37:12,178 ERROR [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Got exception while trying to request the creation of an instance of the "git.project-hobbit.eu:4567/debs_2018_gc/data-generator" image.>

java.lang.NullPointerException: Didn't got a response for a create container message.
at java.util.Objects.requireNonNull(Objects.java:228)
at org.hobbit.core.components.AbstractCommandReceivingComponent.createContainer(AbstractCommandReceivingComponent.java:262)
at org.hobbit.core.components.AbstractCommandReceivingComponent.createContainer(AbstractCommandReceivingComponent.java:222)
at org.hobbit.core.components.AbstractBenchmarkController.createGenerator(AbstractBenchmarkController.java:235)
at org.hobbit.core.components.AbstractBenchmarkController.createDataGenerators(AbstractBenchmarkController.java:196)
at org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController.init(BenchmarkController.java:65)
at org.hobbit.core.run.ComponentStarter.main(ComponentStarter.java:65)
2018-04-23 13:37:12,182 ERROR [org.hobbit.core.components.AbstractBenchmarkController] - <Couldn't create generator component. Aborting.>
2018-04-23 13:37:12,182 ERROR [org.hobbit.core.run.ComponentStarter] -
java.lang.IllegalStateException: Couldn't create generator component. Aborting.
at org.hobbit.core.components.AbstractBenchmarkController.createGenerator(AbstractBenchmarkController.java:241)
at org.hobbit.core.components.AbstractBenchmarkController.createDataGenerators(AbstractBenchmarkController.java:196)
at org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController.init(BenchmarkController.java:65)
at org.hobbit.core.run.ComponentStarter.main(ComponentStarter.java:65)
2018-04-23 13:37:12,185 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <close()>

2018-04-23 15:37:11,986 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-04-23 15:37:11,987 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <DOCKER_CONTAINER_TERMINATED git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller received>

since i'am unable to connect :

docker login git.project-hobbit.eu:4567 -u ******* -p *****
Error response from daemon: Get https://git.project-hobbit.eu:4567/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

It appears that the benchmarkControler is in a public repo but the Data-Generator is private (same for the eval storage).
is it possible to make them all public repositories?

Final results

Hi @smirnp ,

Do we have the final results? I see that the accounts have been restored access. Also there are results available. Are the results final? Those from around GMT: Friday, June 1, 2018 9:13 PM?

Thank you!

Final system registration

Hi @smirnp ,

I have a couple questions on the final system submission. Currently on the conference page it is specified that: If you have an implementation that is ready to participate in the benchmark (required to win the performance award), your solution must be uploaded to the HOBBIT system by the 7th of May (AOE). We will take your latest upload for the final benchmark.

Could you please clarify if there is any other specific step that we need to complete for registering the system for the GC competition - e.g., should we provide you the exact system URI or name and the authors?

Also, how will be the system linked to the corresponding paper submission? I assume we should at least specify somewhere the corresponding paper title/authors.

Maybe this was clarified before, but just to be sure: will it be possible to tune the system parameters after 7th of May (AOE) deadline? and if yes what is the degree of freedom we have for this (e.g., is it possible to submit an updated version of the system?) and until what date?

Thank you,
Emanuel

Feeback of queries

Do we get feedback (like the correct destination and arrival time) for each query?
If not, how does it relate to machine learning prediction?

Online platform execution

Does the Benchmark send a termination message to the System when all tuples are sent? If this is not true, how does our system know that all tuples are sent and when to terminate?

Thanks.

Chance to test solutions

Hi @smirnp,

For the last couple of days the platform was very busy with very long running systems (not related to DEBS). We have systems running for hours. We understand that the platform is a shared resource for various benchmarks, but the current situation makes it close to impossible to be able to proper test our systems before the deadline.

Is it possible to "reserve" the platform for DEBS GC until the deadline? Or other solution you may think of as an "insider"?

Thank you!
Ciprian

delivery errors

Hi all,
I encounter this error, when running "checkHealth",
the tests fail with and without my code
I noticed also that docker /var/lib/docker is full

ERROR [org.hobbit.core.rabbit.DataReceiverImpl] -
com.rabbitmq.client.ShutdownSignalException: clean channel shutdown; protocol method: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)
at com.rabbitmq.client.QueueingConsumer.handle(QueueingConsumer.java:203)
at com.rabbitmq.client.QueueingConsumer.nextDelivery(QueueingConsumer.java:234)
at org.hobbit.core.rabbit.DataReceiverImpl$MsgReceivingTask.run(DataReceiverImpl.java:177)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.rabbitmq.client.ShutdownSignalException: clean channel shutdown; protocol method: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)
at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:554)
at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:509)
at com.rabbitmq.client.impl.ChannelN.close(ChannelN.java:503)
at com.rabbitmq.client.impl.recovery.AutorecoveringChannel.close(AutorecoveringChannel.java:62)
at org.hobbit.core.data.RabbitQueue.close(RabbitQueue.java:90)
at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:303)
at org.hobbit.core.rabbit.DataReceiverImpl.close(DataReceiverImpl.java:122)
at org.apache.commons.io.IOUtils.closeQuietly(IOUtils.java:303)
at org.hobbit.core.components.AbstractTaskGenerator.close(AbstractTaskGenerator.java:269)
at org.hobbit.sdk.utils.ComponentsExecutor.lambda$submit$0(ComponentsExecutor.java:108)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
... 1 more
cheers

Questions about evaluation and query type

Hi, I have several questions regarding the evaluation and online Benchmark:

  1. For the Score A1, will a ship have multiple trips (e.g. A->B and C->D)? If this is true, how to calculate the Score A1 in such case?

  2. Could you give more details on how to calculate the final Score A1? For example, if there are two trips in total and the predicted sequences are:

Trip 1:

  1. Time:01, Predicted Dest: A (Start of Trip)
  2. Time:02, Predicted Dest: B
  3. Time:03, Predicted Dest: A
  4. Time:04, Predicted Dest: B
  5. Time:05, Predicted Dest: B
  6. Time:06, Predicted Dest: B (Arrival at B)

Trip 2:

  1. Time:01, Predicted Dest: C (Start of Trip)
  2. Time:02, Predicted Dest: C
  3. Time:03, Predicted Dest: C
  4. Time:04, Predicted Dest: D
  5. Time:05, Predicted Dest: D (Arrival at D)

What is the final Score A1?

  1. Which one (averageLatencyMs or systemWorkingTimeSeconds or other) is used as the total runtime of the system (Rank B1)?

  2. For the online Benchmark, how to let our system know the current query type (Q1 or Q2) during test?

Thanks!

Empty logs

I run the experiment on the Hobbit platform and the resulting Benchmark and System logs are []. Can you look into this?

Thanks.

Possible docker-compose fill to spin up alternative test setup

Hello

is it possible to spin-up the local test setup without relying on the java sdk / docker builds in code?
I was in the progress of putting together a compose file and this is what I've gotten so far, but I assume there might be some quirks that one might need to now?

version: "3"
services:
  data-generator:
    image: git.project-hobbit.eu:4567/debs_2018_gc/data-generator
    depends_on:
      - rabbit
    environment:
      - QUERY_TYPE=2
      - GENERATOR_LIMIT=72
      - HOBBIT_CONTAINER_NAME=null
      - HOBBIT_GENERATOR_ID=0
      - HOBBIT_RABBIT_HOST=rabbit
      - HOBBIT_GENERATOR_COUNT=1
      - HOBBIT_SESSION_ID=session_1521060139382
      - HOBBIT_GENERATOR_ID=1
      - GENERATOR_TIMEOUT=60
  eval-storage:
    image: git.project-hobbit.eu:4567/debs_2018_gc/eval-storage
    depends_on:
      - rabbit
    environment:
      - QUERY_TYPE=2
      - HOBBIT_EXPERIMENT_URI=http://example.com/exp1
      - SYSTEM_CONTAINER_ID=git.project-hobbit.eu:4567/smirnp/sml-benchmark-v2/system-adapter
      - HOBBIT_CONTAINER_NAME=null
      - HOBBIT_RIAK_NODES=1
      - HOBBIT_RABBIT_HOST=rabbit
      - HOBBIT_SESSION_ID=session_1521060139382
  benchmark-controller:
    image: git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller
    depends_on:
      - rabbit
    environment:
      - HOBBIT_EXPERIMENT_URI=http://example.com/exp1
      - HOBBIT_SESSION_ID=session_1521060139382
      - HOBBIT_CONTAINER_NAME=null
      - HOBBIT_RABBIT_HOST=rabbit
      - 'BENCHMARK_PARAMETERS_MODEL={   "@id" : "http://example.com/exp1",   "generatorLimit" : "72",   "generatorTimeoutMin" : "60",   "queryType" : "2",   "@context" : {     "queryType" : {       "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",       "@type" : "http://www.w3.org/2001/XMLSchema#int"     },     "generatorTimeoutMin" : {       "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorTimeoutMin",       "@type" : "http://www.w3.org/2001/XMLSchema#int"     },     "generatorLimit" : {       "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorLimit",       "@type" : "http://www.w3.org/2001/XMLSchema#int"     }   } }'
  task-generator:
    image: # todo: build locally or have access to image from docker repo
    depends_on:
      - rabbit
    environment:
      - HOBBIT_RABBIT_HOST=rabbit
      - HOBBIT_GENERATOR_ID=0
      - HOBBIT_GENERATOR_COUNT=1
      - GENERATOR_LIMIT=72
      - GENERATOR_TIMEOUT=60
  rabbit:
    image: rabbitmq:latest
    ports:
      - 5672:5672

Issue in training dataset with arrival times

Hello,

We noticed a problem that occurs repeatedly in the training dataset. The arrival time in some ship trips seems to be the time reported by the ship when entering the radius of the first port encountered, and not the time of reaching the reported arrival port. Examples:

  • For ship with id 0x00e225cd5b1926c6b0eb4683692bc2f0ba9a3a2b on a trip from DILISKELESI to ALEXANDRIA the reported arrival time is "26-04-15 20:31". At this time the ship is at (Lon, Lat) = (29.3985, 40.74567), which is the first reported point within the radius of the TUZLA port (not ALEXANDRIA).

  • For ship with id 0x019d2386bff02a0aa96e825e56668dc359e3a987 on a trip from DILISKELESI to GEMLIK the reported arrival time is "12-05-15 4:05". At this time the ship is at (Lon, Lat) = (29.39127, 40.74445), which is the first reported point within the radius of the TUZLA port (not GEMLIK).

  • For ship with id 0x057e6e20d32a9e595926f5f74efce899bad96d97 on a trip from PALMA DE MALLORCA to GENOVA the reported arrival time is "14-05-15 6:01". At this time the ship is at (Lon, Lat) = (7.361688, 43.64136), which is the first reported point within the radius of the MONACO port (not GENOVA).

Should we consider the arrival port, the first port within whose radius the ship entered? or the arrival time is wrong and should be corrected?

We have also noticed some other strange cases in the dataset, such as ship with id 0x023db8bbb80aa3144444cefe369f4e4edfedd70f on a reported trip from PALMA DE MALLORCA to VALENCIA travels this route back and forth several times, but in the dataset this is considered as a single travel. To clarify the issue, the ship arrives to VALENCIA as reported in the dataset at "14-05-15 4:51" (the first point reported within VALENCIA radius). Afterwards looking at the coordinates emitted by the ship, this returns to PALMA DE MALLORCA, but the reported arrival time does not change. Moreover, the departure port and arrival port remains the same (PALMA DE MALLORCA to VALENCIA although the ship moves back from VALENCIA to PALMA DE MALLORCA). Is this behavior intended?

Thank you,

Error while logging in to Hobbit with Docker

Hi Pavel. I currently can't login with the following command:
docker login git.project-hobbit.eu:4567
After entering log in information, I received error message:
Error response from daemon: Get https://git.project-hobbit.eu:4567/v2/: x509: certificate has expired or is not yet valid
My latest login was just about 12 hours ago, and everything worked fine for the past months. Could you please have a look at this issue?
Thank you!

Ranking formula

Hi @smirnp,

Regarding the ranking formula, the site mentions: "the total runtime of the system (Rank B1)." For Rank A1 and A2, the position in the list is used. Can you confirm that the Rank B1 is the total runtime or the position in the list sorted by runtime in increasing order?

If the former is valid, the total runtime, this means that the execution time would have much greater weight in the final score.

Thank you!

Status and deadline

Hi @smirnp ,

Can you:

  1. let us know what is the current status of the benchmarks and what is the plan regarding training and final runs?
  2. Can you confirm that the deadline for tampering with our systems is 2018-05-31 AOE?

Best regards,
Ciprian Amariei

Question about Pairs evaluated

What exactly does Pairs evaluated mean?

I understand that one tuple comes into the queue and the next tuple comes in if we answer that.

That is, if I received 10000 data and processed 10000 data, I think that all tasks should be sequential and evaluated in order.

When I run an online benchmark, if I set more than 10000 tuple limits, the value of Pairs evaluated will be different from the tuple limit I set. At this time, the average earlyness rate of my system drops considerably.

When I run the experiment locally, I set the tuple limit and the Pairs evaluated to the same value up to 100000.
However, these symptoms only occur online.

What am I missing?

Paper format for DEBS Grand Challenge

Hello,

Could you please confirm if the 2-page short paper that should be submitted for describing the GC solution should respect the format/templates of the papers in the main research track ( http://www.cs.otago.ac.nz/debs2018/calls/research.html )? Is there any particular detail we should include?

Also, are we supposed to anonymize the authors as in the research track? I assume not, since the paper should relate with the submitted system anyway. Is that correct or not?

Thank you,
Emanuel

SampleSystemTest exception

I get an error when running mvn test. The OS is Arch linux and I have installed docker and used mvn validate to install required dependencies. Below is the log:

Running org.hobbit.debs_2018_gc_samples.SampleSystemTest
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/root/.m2/repository/org/hobbit/hobbit-java-sdk/1.1.1/hobbit-java-sdk-1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/root/.m2/repository/org/slf4j/slf4j-log4j12/1.7.15/slf4j-log4j12-1.7.15.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-02-05 17:11:05,507 DEBUG [Jena] - <Jena initialization>
2018-02-05 17:11:06,489 ERROR [RabbitMqDockerizer] - <Exception: {}>
com.spotify.docker.client.exceptions.DockerException: java.util.concurrent.ExecutionException: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
	at com.spotify.docker.client.DefaultDockerClient.propagate(DefaultDockerClient.java:2302)
	at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2223)
	at com.spotify.docker.client.DefaultDockerClient.listContainers(DefaultDockerClient.java:526)
	at org.hobbit.sdk.docker.AbstractDockerizer.findContainersByName(AbstractDockerizer.java:374)
	at org.hobbit.sdk.docker.AbstractDockerizer.createContainerIfNotExists(AbstractDockerizer.java:161)
	at org.hobbit.sdk.docker.AbstractDockerizer.run(AbstractDockerizer.java:71)
	at org.hobbit.sdk.docker.RabbitMqDockerizer.run(RabbitMqDockerizer.java:36)
	at org.hobbit.debs_2018_gc_samples.SampleSystemTest.checkHealth(SampleSystemTest.java:103)
	at org.hobbit.debs_2018_gc_samples.SampleSystemTest.checkHealth(SampleSystemTest.java:66)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:115)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:75)
Caused by: java.util.concurrent.ExecutionException: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
	at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture$Sync.getValue(AbstractFuture.java:299)
	at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture$Sync.get(AbstractFuture.java:286)
	at jersey.repackaged.com.google.common.util.concurrent.AbstractFuture.get(AbstractFuture.java:116)
	at com.spotify.docker.client.DefaultDockerClient.request(DefaultDockerClient.java:2221)
	... 36 more
Caused by: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory
	at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:481)
	at org.glassfish.jersey.apache.connector.ApacheConnector$1.run(ApacheConnector.java:491)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at jersey.repackaged.com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
	at java.util.concurrent.AbstractExecutorService.submit(AbstractExecutorService.java:112)
	at jersey.repackaged.com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:50)
	at jersey.repackaged.com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:37)
	at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:487)
	at org.glassfish.jersey.client.ClientRuntime$2.run(ClientRuntime.java:178)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
	at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:340)
	at org.glassfish.jersey.client.ClientRuntime$3.run(ClientRuntime.java:210)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: No such file or directory
	at jnr.unixsocket.UnixSocketChannel.doConnect(UnixSocketChannel.java:94)
	at jnr.unixsocket.UnixSocketChannel.connect(UnixSocketChannel.java:102)
	at com.spotify.docker.client.ApacheUnixSocket.connect(ApacheUnixSocket.java:76)
	at com.spotify.docker.client.UnixConnectionSocketFactory.connectSocket(UnixConnectionSocketFactory.java:78)
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
	at org.glassfish.jersey.apache.connector.ApacheConnector.apply(ApacheConnector.java:435)

evaluation

Hi all,
we can evaluate the system using the training dataset itself,
but I would like to use a different file, for sending test streams
the available files include unknown ports not included in the training dataset
best,

Response lost while sending to EvalStorage

I am currently having the issue of timeout (set to 60mins) error due to the fact that Actuals is less than Expectation. So I made a number of tries on your Original code. I happed to me that when the GENERATOR_LIMIT is small (say 72, 1k or 10k), the issue did not occurs. But when this threshold is set to 100k, it got stuck with the following log until timeout:

DEBUG [eval-storage-dockerizer] - <2018-03-29 16:55:23,985 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 99753 (0 t/s) Actuals: 99751 (0 t/s)>

I tested on 2 machines, and the above mentioned issue happened in both cases (expectation is always 99753 and actuals is 99751). In the code involving the solution, the issue occurs at earlier tuples. Additionally, I did stop and remove any running containers before starting the new test.

This is my Java and Docer version:

java version "1.8.0_162"
Java(TM) SE Runtime Environment (build 1.8.0_162-b12)
Java HotSpot(TM) 64-Bit Server VM (build 25.162-b12, mixed mode)

Docker version 18.03.0-ce, build 0520e24

Could you please reproduce the issue and try to help me solve it? Thank you very much.

Similiarity of training and evaluation / test data

I've been wondering if you can / want to provide some information on how similar the training data is compared to the test data that the systems get evaluated with? It would be especially interesting would to know whether the ports in the evaluation are restricted to those 20 ports that are provided as training data, or the global list with all ports worldwide?

SampleSystemTest::checkHealth sometimes not working / SystemAdapter code not executed

Hey everyone,

I have a problem where the checkHealth test sometimes does not seem to execute the SystemAdapter code at all, but instead gets stuck in a loop of

2018-03-28 17:02:51,362 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:52,248 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:52,251 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>

A already put together a script to clean up all remnants of previous runs, but so far I was not able to see in which cases this works / in which cases it doesn't.

My cleanup steps

mvn clean package -DskipTests
docker ps -a | grep "java -cp" | awk '{ print $1 }' | xargs docker rm -f && docker rm -f rabbit
docker network rm hobbit-common
docker network rm hobbit

Please note that sometimes it does work (meaning I also see the accuracy score, logs, ...), it's just that I can't figure out what is necessary to have it work every time. I attached the full logs of that run below. Any help would be very much appreciated

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/florianschmidt/.m2/repository/org/hobbit/hobbit-java-sdk/1.1.1/hobbit-java-sdk-1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/florianschmidt/.m2/repository/org/slf4j/slf4j-log4j12/1.7.15/slf4j-log4j12-1.7.15.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-03-28 17:02:23,970 DEBUG [Jena] - <Jena initialization>
2018-03-28 17:02:26,938 DEBUG [RabbitMqDockerizer] - <Creating container (imageName=rabbitmq:latest)>
2018-03-28 17:02:27,228 DEBUG [RabbitMqDockerizer] - <Starting container (imageName=rabbitmq:latest, containerId=bd8548b843209a0ee833c9520dd1f687c64568b01e84d8ba0c1a150d43ba31b0)>
2018-03-28 17:02:27,667 DEBUG [RabbitMqDockerizer] - <Connecting container to networks (imageName=rabbitmq:latest)>
2018-03-28 17:02:28,436 DEBUG [RabbitMqDockerizer] - <Container started (imageName=rabbitmq:latest, containerId=bd8548b843209a0ee833c9520dd1f687c64568b01e84d8ba0c1a150d43ba31b0)>
2018-03-28 17:02:28,436 DEBUG [RabbitMqDockerizer] - <Starting monitoring & logs reading for container (imageName=rabbitmq:latest)>
2018-03-28 17:02:31,443 DEBUG [org.hobbit.sdk.docker.RabbitMqDockerizer] - <Trying to connect to container at rabbit (imageName=rabbitmq:latest)>
2018-03-28 17:02:37,643 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing CommandQueueListener>
2018-03-28 17:02:37,643 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initializing...>
2018-03-28 17:02:37,714 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:37,714 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running CommandQueueListener>
2018-03-28 17:02:37,714 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initialized. Waiting for termination signal>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing benchmark-controller-dockerizer>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running benchmark-controller-dockerizer>
2018-03-28 17:02:37,716 DEBUG [benchmark-controller-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing SystemAdapter>
2018-03-28 17:02:37,804 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:37,816 DEBUG [benchmark-controller-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:37,874 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter] - <Init()>
2018-03-28 17:02:37,878 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter] - <SystemModel: {
  "@id" : "http://jenaKeyValue.com/experimentUri",
  "queryType" : "2",
  "@context" : {
    "queryType" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    }
  }
}
>
2018-03-28 17:02:37,878 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running SystemAdapter>
2018-03-28 17:02:37,939 DEBUG [benchmark-controller-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=720c72ca57c6cfb452f1ce662e4192ada69d0be7069eb60848301dba13aab104)>
2018-03-28 17:02:38,329 DEBUG [benchmark-controller-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:38,685 DEBUG [benchmark-controller-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=720c72ca57c6cfb452f1ce662e4192ada69d0be7069eb60848301dba13aab104)>
2018-03-28 17:02:38,685 DEBUG [benchmark-controller-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:39,948 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:40,005 DEBUG [Jena] - <Jena initialization>
>
2018-03-28 17:02:40,366 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator>
2018-03-28 17:02:40,367 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing data-generator-dockerizer>
2018-03-28 17:02:40,367 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running data-generator-dockerizer>
2018-03-28 17:02:40,367 DEBUG [data-generator-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:40,441 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,463 DEBUG [data-generator-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:40,473 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/task-generator>
2018-03-28 17:02:40,475 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing TaskGenerator>
2018-03-28 17:02:40,540 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,542 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,572 DEBUG [data-generator-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator, containerId=b7c9cb815a5ff877cfdf67c23758d11165ce907f240d64b3f63884940274d7ae)>
2018-03-28 17:02:40,603 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Init()>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <GENERATOR_LIMIT=72>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <GENERATOR_TIMEOUT=60>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Init (queryType=0, recordsLimit=72, timeout=60)>
2018-03-28 17:02:40,639 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Reading data/debs2018_training_labeled.csv>
2018-03-28 17:02:40,993 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:41,016 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Init()>
2018-03-28 15:02:41,068 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <BenchmarkModel: {
  "@id" : "http://jenaKeyValue.com/experimentUri",
  "generatorLimit" : "72",
  "generatorTimeoutMin" : "60",
  "queryType" : "2",
  "@context" : {
    "generatorLimit" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorLimit",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    },
    "generatorTimeoutMin" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorTimeoutMin",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    },
    "queryType" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    }
  }
}
>
2018-03-28 15:02:41,069 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Gen.EnvVariables: QUERY_TYPE=2, GENERATOR_LIMIT=72, GENERATOR_TIMEOUT=60>
2018-03-28 15:02:41,069 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createDataGenerators()>
2018-03-28 15:02:41,183 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createTaskGenerators()>
2018-03-28 15:02:41,278 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForComponents()>
2018-03-28 15:02:41,279 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received EVAL_STORAGE_READY_SIGNAL>
>
2018-03-28 17:02:41,524 DEBUG [data-generator-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:41,537 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <File reading finished>
2018-03-28 17:02:41,538 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Processing 73 lines>
2018-03-28 17:02:41,547 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Processing finished: 72 tuples, 2 trips>
2018-03-28 17:02:41,553 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running TaskGenerator>
2018-03-28 17:02:41,928 DEBUG [data-generator-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator, containerId=b7c9cb815a5ff877cfdf67c23758d11165ce907f240d64b3f63884940274d7ae)>
2018-03-28 17:02:41,928 DEBUG [data-generator-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:42,012 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:42,293 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received TASK_GENERATOR_READY_SIGNAL>
>
2018-03-28 17:02:42,089 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:42,104 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage>
2018-03-28 17:02:42,105 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing eval-storage-dockerizer>
2018-03-28 17:02:42,106 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running eval-storage-dockerizer>
2018-03-28 17:02:42,106 DEBUG [eval-storage-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:42,283 DEBUG [eval-storage-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:42,284 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:42,291 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to start.>
2018-03-28 17:02:42,340 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <generateTask()>
2018-03-28 17:02:42,341 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Start sending tuples(2 ships)>
2018-03-28 17:02:42,375 DEBUG [eval-storage-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage, containerId=f76d6a74a351189bf6102f9f4e9e704c09f49068733c60e1e0884123d092a3b0)>
2018-03-28 17:02:42,753 DEBUG [eval-storage-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:43,026 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:42,773 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received DATA_GENERATOR_READY_SIGNAL>
2018-03-28 15:02:42,826 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createEvaluationStorage()>
2018-03-28 15:02:43,023 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <executeBenchmark(sending TASK_GENERATOR_START_SIGNAL & DATA_GENERATOR_START_SIGNAL)>
2018-03-28 15:02:43,023 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForDataGenToFinish() to send DATA_GENERATION_FINISHED_SIGNAL>
2018-03-28 15:02:43,025 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 1 Data Generators to finish.>
>
2018-03-28 17:02:43,047 DEBUG [eval-storage-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage, containerId=f76d6a74a351189bf6102f9f4e9e704c09f49068733c60e1e0884123d092a3b0)>
2018-03-28 17:02:43,047 DEBUG [eval-storage-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:43,344 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 2 tuples/s>
2018-03-28 17:02:44,048 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:44,243 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received EVAL_STORAGE_READY_SIGNAL>
>
2018-03-28 17:02:44,093 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:44,207 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Init()>
2018-03-28 15:02:44,235 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Init (queryType=2, experimentUri=http://example.com/exp1, systemContainerId=git.project-hobbit.eu:4567/smirnp/sml-benchmark-v2/system-adapter>
>
2018-03-28 17:02:44,348 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:45,114 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:45,241 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (2 t/s) Actuals: 2 (2 t/s)>
>
2018-03-28 17:02:45,348 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:45,607 DEBUG [data-generator-dockerizer] - <2018-03-28 15:02:45,300 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.DataGenerator] - <close()>
>
2018-03-28 17:02:46,136 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:46,241 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:46,353 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:46,664 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:46,693 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to finish.>
2018-03-28 17:02:46,750 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:46,751 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to finish.>
2018-03-28 17:02:47,106 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:47,414 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForTaskGenToFinish() to finish to send TASK_GENERATION_FINISHED_SIGNAL>
2018-03-28 15:02:47,415 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 1 Task Generators to finish.>
>
2018-03-28 17:02:47,158 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:47,244 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:47,355 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:48,174 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:48,245 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:48,352 DEBUG [org.hobbit.core.rabbit.DataReceiverImpl] - <Receiver task terminates after receiving 1 messages.>
2018-03-28 17:02:48,359 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:49,198 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:49,246 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:49,360 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:50,214 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:50,248 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:50,361 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:51,230 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:51,250 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:51,362 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:52,248 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:52,251 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:52,367 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:53,277 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:53,254 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:53,371 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:54,297 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:54,255 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:54,376 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:55,317 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:55,255 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:55,378 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
Disconnected from the target VM, address: '127.0.0.1:61273', transport: 'socket'

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)

Questions about online benchmark

I have three questions about the online benchmark:

  1. Which queue (task queue or data queue) is used by the benchmark to send the query to our system?

  2. Should our system connect to both the task queue and data queue or just the task queue?

  3. According to the local benchmark, the output format is "port name, arrival time". Shouldn't we put the task id in the output?

Thanks!

Dataset Availability

Hi guys,
do you intend to provide a complete benchmark dataset of marine traffic for this challenge? If so, by when we should be expecting that it will be available?

Thanks

Missing experiments/results

Hi @smirnp ,

The context: we have some parameters set for the system. Between runs, we have noticed that the system is updated a little bit later probably due to some cache/lag. In order to make sure we run the experiment with the new changed parameters, we have also (multiple times) changed the system's URI and label.

Recently, we have noticed that our system experiments are not saved/reported. Here are some examples:
https://master.project-hobbit.eu/experiments/1525518667343
https://master.project-hobbit.eu/experiments/1525518463087
https://master.project-hobbit.eu/experiments/1525519056988

They do not display the outcome and they cannot be found in results section.

It is weekend, but if you manage to see this, I really hope for a quick turnaround given the deadline.

Thanks a lot for your effort!

Training dataset with blank fields

Hi!
In the last labeled training dataset, pushed on 05/02/2017, there are 3564 tuples without arrival_cal and arrival_calc_port. It's an error or missed data?

thanks,
Michele

Arrival Port Prediction

I have a question about the arrival prediction of the port.

For ships with the ship id of "0x0b98f267e96f76f14e1525a40df0274066124eb5", it will travel to BARCELONA -> unknown port -> MONACO -> LIVORNO. (Inside the port radius)

In this case, all depart ports are BARCELONA.

I knew that we decided to split this trip. But it did not actually separate.

I am confused that the depart port does not change when entering a port radius of unknown port, MONACO, or LIVORNO.

  1. First, what is the prediction for BARCELONA -> unknown port?
  2. When going from unknown port to MONACO, is the arrival port predicted by MONACO?
    Or is LIVORNO the correct prediction?

Could you explain more about this algorithm?

Golden solution?

We have successfully got the local Benchmark up and running. The question is: how do we get the golden solution for each query? We didn't receive any feedback (correct destination and arrival time) from the Benchmark after we send the prediction result. We assume the contest is running in a online fashion as you replied here.

Thanks!

Draught invalid value?

Hi @smirnp

We have the noticed a strange value for draught, i.e. "" (two double quotes). Can we expect this for other fields also?

Thank you!

Q2 ranking

@smirnp, for Q2 does the port prediction have any influence on it? It is not clear from description if the port prediction is relevant for Q2 - the descriptions only mentions about the time.

Thank you!

Questions about the dataset

Hi, we have several questions about the dataset:

  1. Will a tuple be sent (queried) more than once?
  2. What is the maximum possible number of tasks to handle by our system?
  3. In the provided training data, many tuples have the same SHIP_ID and ARRIVAL_PORT. As a result, the destination port is largely dependent on the SHIP_ID. Is this normal?
  4. What is the maximum possible number of ships/SHIP_ID (unique ship)?
  5. Will we know the range of every field?

Thanks.

Get evaluation results of a solution in local machine

Hi guys,

I ran the provided code on my machine and implemented a solution. It produces an output which will be sent to evaluation storage. Then, I want to know how good my solution is.

Does anyone know how we can get evaluation result, i.e. output of the evaluation module?

Thanks!

Best,
Duy

System erroring on hobbit platform

When running the system on the hobbit platform our experiment is shown as terminated with an error. Running the main file locally similar to the python-example works fine locally.
We are using python sample and the following Dockerfile

FROM python:3.6-slim
COPY . /usr/src/app/
RUN pip install 'scikit-learn==0.18.1' pika numpy scipy pandas bitstring
CMD ["python", "/usr/src/app/main.py"]

I am having a hard time debugging the issue because the System Log is completely empty. The Benchmark log shows that it starts running and then terminates after timeout, followed by a vast number of RabbitMqShutdownSignal Exceptions.

Any help / hints would be greatly appreciated!

The benchmark logs (a little trimmed)

"Caused by: com.rabbitmq.client.ShutdownSignalException: clean channel shutdown; protocol method: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)"
"\tat java.lang.Thread.run(Thread.java:745)"
"\tat org.hobbit.core.rabbit.DataReceiverImpl$MsgReceivingTask.run(DataReceiverImpl.java:177)"
"\tat com.rabbitmq.client.QueueingConsumer.nextDelivery(QueueingConsumer.java:234)"
"\tat com.rabbitmq.client.QueueingConsumer.handle(QueueingConsumer.java:203)"
"com.rabbitmq.client.ShutdownSignalException: clean channel shutdown; protocol method: #method<channel.close>(reply-code=200, reply-text=OK, class-id=0, method-id=0)"
"2018-05-06 11:01:51,741 ERROR [org.hobbit.core.rabbit.DataReceiverImpl] - <Exception while waiting for delivery.>"
"2018-05-06 11:01:51,732 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <close()>"
"2018-05-06 11:01:43,705 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <close()>"
"2018-05-06 11:01:43,698 ERROR [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Timeout reached. Stopping benchmark>"
"2018-05-06 10:56:42,685 INFO [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Benchmark is ready. Waiting for the system init>"
"2018-05-06 10:56:42,685 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createTaskGenerators(1)>"
"2018-05-06 10:56:42,684 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received EVAL_STORAGE_READY_SIGNAL>"
"2018-05-06 10:56:42,679 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Sun May 06 10:56:42 UTC 2018 Init (queryType=2, experimentUri=http://w3id.org/hobbit/experiments#1525604184496>"
"2018-05-06 10:56:42,658 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Init()>"
"2018-05-06 10:56:42,452 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for Evaluation Storage to be ready.>"
"2018-05-06 10:56:42,451 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 0 Data Generators to be ready.>"
"2018-05-06 10:56:42,451 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 1 Data Generators to be ready.>"
"2018-05-06 10:56:39,892 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received TASK_GENERATOR_READY_SIGNAL>"
"2018-05-06 10:56:39,883 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <Processing finished: 2000 tuples, 1874 labeled, 3 ships, 8 trips, 3 labeled>"
"2018-05-06 10:56:39,782 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <Processing 2001 lines>"
"2018-05-06 10:56:39,781 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <File reading finished (2001)>"
"2018-05-06 10:56:39,697 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <Reading data/debs2018_fixed_3_labeled_v7_test_20.csv>"
"2018-05-06 10:56:39,683 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <Init (genId=0, queryType=2, recordsLimit=2000>"
"2018-05-06 10:56:39,683 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <TUPLES_LIMIT=2000>"
"2018-05-06 10:56:39,671 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.TaskGenerator_0] - <Init()>"
"2018-05-06 10:56:36,406 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Gen.EnvVariables: HOBBIT_EXPERIMENT_URI=http://w3id.org/hobbit/experiments#1525604184496, QUERY_TYPE=2, TUPLES_LIMIT=2000, HOBBIT_GENERATOR_ID=1, HOBBIT_GENERATOR_COUNT=1>"
"2018-05-06 10:56:36,407 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createTaskGenerators(1)>"
">"
"}"
"  }"
"    }"
"      \"@id\" : \"http://project-hobbit.eu/sml-benchmark-v2/tuplesLimit\","
"      \"@type\" : \"http://www.w3.org/2001/XMLSchema#int\""
"    },"
"    \"tuplesLimit\" : {"
"      \"@type\" : \"http://www.w3.org/2001/XMLSchema#int\""
"      \"@id\" : \"http://project-hobbit.eu/sml-benchmark-v2/queryType\","
"    },"
"    \"queryType\" : {"
"      \"@type\" : \"http://www.w3.org/2001/XMLSchema#int\""
"      \"@id\" : \"http://project-hobbit.eu/sml-benchmark-v2/benchmarkTimeoutMin\","
"    \"benchmarkTimeoutMin\" : {"
"  \"@context\" : {"
"  \"tuplesLimit\" : \"2000\","
"  \"benchmarkTimeoutMin\" : \"5\","
"  \"@id\" : \"http://w3id.org/hobbit/experiments#New\","
"  \"queryType\" : \"2\","
"2018-05-06 10:56:36,405 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <BenchmarkModel: {"
"2018-05-06 10:56:36,380 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Init()>"
"2018-05-06 10:56:35,824 DEBUG [Jena] - <Jena initialization>"```

Coordinate format of sample data

Which coordinate system are lat / lon specified in? If they are in WGS84 Decimal Degrees, I assume there is a decimal point to be added after two digits?

VESSEL_ID;speed;LON;LAT;COURSE;HEADING;TIMESTAMP;departurePortName;Reported_Draught
0x0fe9acdb3675a8a2942fafbd4af61bc37e44c0ec;45;1474895;4066721;197;223;30/5/2015 4:35;SALERNO;88

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.