Need help to reproduce results about rel HOT 10 CLOSED

Hi @vardaan123 ,

One immediate thing I see at [0] is that you are not using strong, but weak annotation matching. I have attached our GERBIL platform run [1] I observe that GERBIL updated its version, which could be the cause of some minor differences, but hopefully not the huge difference you're mentioning.

We did some reformatting in the code and it seems that the reference in the GERBIL middleware to the entity was incorrect (sorry about that). Could you please pull the latest version of REL [2]?

Let me know if this fixes your problem :)!

[0] http://gerbil.aksw.org/gerbil/experiment?id=202010250000
[1] http://gerbil.aksw.org/gerbil/experiment?id=202002140003
[2] e5a13af

from rel.

Thanks for pointing out the mistake. I also found that I was using AIDA as the annotator instead of the URI (I didn't click on the Add Annotator button). I am now running it again. But I get the error on my GERBIL server

org.apache.http.conn.HttpHostConnectException: Connect to localhost:5555 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused 

Not sure why this happens. Both port 1235 and 5555 are open on my remote server which are used to host the API and the GERBIL server.

from rel.

Funny enough I made that exact same mistake when I first started working with REL!

Not sure why you are getting this error, but the only time I got that specific error is when my ports were not opened correctly. I am unsure how I can help you with this as it seems to be unrelated to REL. Sorry for the late response, hope you've had some luck in the meantime?

from rel.

Actually, your error regards the port on localhost. Either modify the IP to be connected to, or tunnel the remote port to localhost (using ssh - L for example, or putty).

from rel.

Thanks for your suggestions. I was able to fix this issue. I have a question about the D2KB results. Do you get the D2KB results from the D2KB row in A2KB task or do you run a separate D2KB experiment on the GERBIL server? I tried both but there seems to be a difference between both of these. For the results URI you provided (http://gerbil.aksw.org/gerbil/experiment?id=202002140003), the ED numbers in Table 2 differ a lot (85.5 vs 83.58 Macro F1 for AIDA-B) and (86.6 vs 85.14 Micro F1 for AIDA-B).

from rel.

Hi Vardaan,

We get the results by running a separate D2KB experiment on the GERBIL server because the D2KB row for an A2KB run is dependent on the mentions you were able to retrieve in the step prior to that (see [0] for our D2KB run on GERBIL).

Nonetheless, it could be that you observe a difference in performance across runs for different GERBIL versions. This is, unfortunately, something we cannot debug as GERBIL does not provide us with any feedback. One thing we do to verify that our modules perform consistently is to perform evaluate our results locally (see Table 3). I performed another local run for the Wikipedia 2019 corpus (see Table 3) and can confirm that the results are consistent with what we published in our paper. If you'd like to perform a local run yourself, I'd like to refer you to [1] (run python -m scripts.code_tutorials.train_eval_ED).

Let me know if anything is unclear and thanks for using REL :)!

[0] http://gerbil.aksw.org/gerbil/experiment?id=202002130001
[1] https://github.com/informagi/REL/blob/master/scripts/code_tutorials/train_eval_ED.py

from rel.

Thanks a lot for clarifying my doubts!

from rel.

Hi @vardaan123, when you verify on the remote server, Is the first step mvn clean -Dmaven.tomcat.port=1235 tomcat:run ?
And then you run api.py, right?
On GERBIL experiment configuration, the URI is http://{remote_ADD}:1235/gerbil-spotWrapNifWS4Test/myalgorithm，right?
When you click on the Add Annotator button, do you receive error Warning! There was an error while testing the annotator. Exception while sending request.?
I always get stuck in this step. Could you give me some suggestions? Thanks!

from rel.

from rel.

Hi @BeerTai, did you successfully fix that problem? I am currently experiencing that same problem you had: Warning! There was an error while testing the annotator. Exception while sending request.

If so, could you please indicate to me how you fix it?
Thanks!

from rel.