informagi / rel Goto Github PK

REL: Radboud Entity Linker

License: MIT License

Python 91.43% Dockerfile 0.16% Makefile 0.06% Shell 0.31% Java 8.04%

rel's Introduction

REL: Radboud Entity Linker

REL is a modular Entity Linking package that is provided as a Python package as well as a web API. REL has various meanings - one might first notice that it stands for relation, which is a suiting name for the problems that can be tackled with this package. Additionally, in Dutch a 'rel' means a disturbance of the public order, which is exactly what we aim to achieve with the release of this package.

REL utilizes English Wikipedia as a knowledge base and can be used for the following tasks:

Entity linking (EL): Given a text, the system outputs a list of mention-entity pairs, where each mention is a n-gram from text and each entity is an entity in the knowledge base.
Entity Disambiguation (ED): Given a text and a list of mentions, the system assigns an entity (or NIL) to each mention.

Documentation available at https://rel.readthedocs.io

Suggestions, improvements, and edits are most welcome.

Calling our API

Users may access our API by using the example script below. For EL, the spans field needs to be set to an empty list. For ED, however, the spans field should consist of a list of tuples, where each tuple refers to the start position and length of a mention.

import requests

API_URL = "https://rel.cs.ru.nl/api"
text_doc = "If you're going to try, go all the way - Charles Bukowski"

# Example EL.
el_result = requests.post(API_URL, json={
    "text": text_doc,
    "spans": []
}).json()

# Example ED.
ed_result = requests.post(API_URL, json={
    "text": text_doc,
    "spans": [(41, 16)]
}).json()

Using REL as a Python package

You can also use REL as a Python package. See the hello world example below for how to get started. For more examples, have a look at the documentation.

from REL.mention_detection import MentionDetection
from REL.utils import process_results
from REL.entity_disambiguation import EntityDisambiguation
from REL.ner import Cmns, load_flair_ner

wiki_version = "wiki_2014"
base_url = "C:/path/to/rel_data"

input_text = {
    "my_doc": ("Hello, world!", []),
}

mention_detection = MentionDetection(base_url, wiki_version)
tagger_ner = load_flair_ner("ner-fast")

tagger_ngram = Cmns(base_url, wiki_version, n=5)
mentions, n_mentions = mention_detection.find_mentions(input_text, tagger_ngram)

config = {
    "mode": "eval",
    "model_path": "ed-wiki-2014",
}
model = EntityDisambiguation(base_url, wiki_version, config)

predictions, timing = model.predict(mentions)
result = process_results(mentions, predictions, input_text)
print(result)
# {'my_doc': [(0, 13, 'Hello, world!', 'Hello_world_program', 0.6534378618767961, 182, '#NGRAM#')]}

Installation

This section describes how to deploy REL on a local machine and setup the API. If you want to do anything more than simply running our API locally, you can skip the Docker steps and continue with installation from source.

Option 1: Installation using pip

pip install radboud-el

Option 2: Installation using Docker

First, download the necessary data; you need the generic files and a Wikipedia version (2014 or 2019) (see Download). Extract them anywhere, we will bind the directories to the Docker container as volumes.

./scripts/download_data.sh ./data generic wiki_2019

Prebuilt images

To use our prebuilt default image, run:

docker pull informagi/rel

To run the API locally:

# Map container port 5555 to local port 5555, and use Wikipedia 2019
# Also map the generic and wiki_2019 folders to directories in Docker container
docker run \
    -p 5555:5555 \
    -v $PWD/data/:/workspace/data \
    --rm -it informagi/rel \
    python -m REL.server --bind 0.0.0.0 --port 5555 /workspace/data wiki_2019

Now you can make requests to http://localhost:5555 (or another port if you use a different mapping) in the format described in the example above.

Build your own Docker image

To build the Docker image yourself, run:

# Clone the repository
git clone https://github.com/informagi/REL && cd REL
# Build the Docker image
docker build . -t informagi/rel

To run the API locally, use the same commands as mentioned in the previous section.

Option 3: Installation from source code

Run the following command in a terminal to install REL:

pip install git+https://github.com/informagi/REL

You will also need to manually download the files described in the next section.

Download data

The files used for this project can be divided into three categories. The first is a generic set of documents and embeddings that was used throughout the project. This folder includes the GloVe embeddings and the unprocessed datasets that were used to train the ED model. The second and third category are Wikipedia corpus related files, which in our case either originate from a 2014 or 2019 corpus. Alternatively, users may use their own corpus, for which we refer to the tutorials.

Tutorials

To promote usage of this package we developed various tutorials. If you simply want to use our API, then we refer to the section above. If you feel one is missing or unclear, then please create an issue, which is much appreciated :)!

The first two tutorials are for users who simply want to use our package for EL/ED and will be using the data files that we provide. The remainder of the tutorials are optional and for users who wish to e.g. train their own Embeddings.

REL variants

REL comes in two variants for identifying entity mentions:

Case-sensitive: This setup is suitable for properly written texts (e.g., news articles) and is the default setup of the REL package. In this setup, we use the ner-fast FLAIR model, which is case-sensitive. The results reported in the REL paper are based on this model.
Case-insensitive: This setup is well suited for noisy texts (e.g., queries), where entity mentions can be (often) lowercased. In this setup, we use the ner-fast-with-lowercase model, which is the ner-fast FLAIR architucture trained on randomly cased and uncased text. This variant is the default setup of our API.

Below is a comparison of these two models on CoNLL-2003 NER dataset.

Model	CoNLL-2003 test	F1
`ner-fast`	original	92.78
`ner-fast`	lower-cased	58.42
`ner-fast`	random	70.64
`ner-fast-with-lowercase`	original	91.53
`ner-fast-with-lowercase`	lower-cased	89.73
`ner-fast-with-lowercase`	random	89.66

See Notes on using custom models for further information on switiching between these variants.

Efficiency of REL

We measured the efficiency of REL on a per-document basis. We ran our API with 50 documents from AIDA-B with > 200 words, which is 323 (± 105) words and 42 (± 19) mentions per document. The results are added to the table below.

Model	Time MD	Time ED
With GPU	0.44±0.22	0.24±0.08
Without GPU	2.41±1.24	0.18±0.09

As our package has changed overtime, we refer to one of our earlier commits for reproducing the results in the table above. To reproduce the results above, perform the following steps:

Start the server. As can be seen in server.py, we added checkpoints in our server calls to measure time taken per call.
Once the server is started, run the efficiency test. Do not forget to update the base_url to specify where the data is located in the filesystem. This directory refers to where all project-related data is stored (see our tutorial on how to get started
Finally, process the efficiency results.

Development

Check out our Contributing Guidelines to get started with development.

Cite

If you are using REL, please cite the following paper:

@inproceedings{vanHulst:2020:REL,
 author =    {van Hulst, Johannes M. and Hasibi, Faegheh and Dercksen, Koen and Balog, Krisztian and de Vries, Arjen P.},
 title =     {REL: An Entity Linker Standing on the Shoulders of Giants},
 booktitle = {Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval},
 series =    {SIGIR '20},
 year =      {2020},
 publisher = {ACM}
}

Contact

If you find any bugs or experience difficulties when using REL, please create a issue on this Github page. If you have any specific questions with respect to our research with REL, please email Faegheh Hasibi.

Acknowledgements

Our thanks go out to the authors that open-sourced their code, enabling us to create this package that can hopefully be of service to many.

rel's People

Contributors

Stargazers

Watchers

rel's Issues

Option to return the flair entity types in the response body

FLAIR labels the detected mentions with an entity type. We could return these "for free", instead of requiring an extra join against DBPedia / Wikidata.

Question about mention-entity prior P(e|m)

Hi, thanks for your project! I have a question about P(e|m).
In the code, we can get e and m by extracting text <a href=entity>mention</a>. Finally we generate a mention-entity mapping table.
My question is if we recognize a mention from a text, however, the mention may not exist in the mapping table. How do we calculate the prior of the mention？
Can we guarantee that the mention after entity recognition must exist in the mapping table?
Thanks！

API for EL seems not working

Hi all,

I copy-paste the commands in the tutorial for API usage, but I got error when using EL api. ED seems to be fine. The response I got is HTTP/1.0 400 Bad Request\r\nServer: BaseHTTP/0.6 Python/3.7.3\r\nDate: Tue, 28 Jul 2020 18:35:13 GMT\r\n\r\n.

I am wondering if something changed in the server caused this?

Trying to set up with a local endpoint

Hello, I downloaded the files and I am trying to run the run_server.py on a linux environment to setup a local endpoint. I get this error :
2021-03-31 14:48:58.259298: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library libcudart.so.10.1
I0331 14:49:00.938567 139779044779328 textcleaner.py:37] 'pattern' package not found; tag filters are not available for English
Traceback (most recent call last):
File "run_server.py", line 18, in
model = EntityDisambiguation(base_url, wiki_version, config)
File "/home/gv1020/.local/lib/python3.6/site-packages/REL/entity_disambiguation.py", line 49, in init
self.g_emb = GenericLookup("common_drawl", os.path.join(base_url, "generic"))
File "/home/gv1020/.local/lib/python3.6/site-packages/REL/db/generic.py", line 36, in init
self.db = self.initialize_db(path_db, table_name, columns)
File "/home/gv1020/.local/lib/python3.6/site-packages/REL/db/base.py", line 41, in initialize_db
db = sqlite3.connect(fname, isolation_level=None)
sqlite3.OperationalError: unable to open database file

I have installed pattern package as well. Is this something you can help me with? Thanks!

Model for ED confidence score

Hi, where can I find the code to train a model to estimate posterior probabilities of the linked entities (lr_model.pkl). Or could you briefly describe how to train such a model?

`lr_model.pkl` for ED2019

I noticed that the lr_model.pkl file is missing in the ED2019 model archive. Is it possible to provide this model as well, since there are a quite a few false positives detected otherwise, with no way to filter out low confidence entities.

For instance, (from the documentation):Obama will visit Germany. And have a meeting with Merkel tomorrow gives me the following entities with the 2019 model:

[(0, 5, 'Obama', 'Barack_Obama', 0.0, 13399, '#NGRAM#'),
  (6, 4, 'will', 'Will_and_testament', 0.0, 9281, '#NGRAM#'),
  (11, 5, 'visit', 'State_visit', 0.0, 2423, '#NGRAM#'),
  (17, 8, 'Germany.', 'Germany', 0.0, 612, '#NGRAM#'),
  (9, 9, 'a meeting', 'Trump_Tower_meeting', 0.0, 6, '#NGRAM#'),
  (0, 3, 'And', 'Ehud_Olmert', 0.0, 1053, '#NGRAM#'),
  (4, 4, 'have', 'Captain_America_(1990_film)', 0.0, 1988, '#NGRAM#'),
  (19, 4, 'with', 'Two-round_system', 0.0, 1498, '#NGRAM#'),
  (24, 6, 'Merkel', 'Angela_Merkel', 0.0, 1499, '#NGRAM#'),
  (31, 9, 'tomorrow.', 'April_20', 0.0, 25, '#NGRAM#')]

Error in running tutorial2

I get following errors when running tagger_ner = SequenceTagger.load("ner-fast") in tutorial 2 on a server.

Traceback (most recent call last):
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connection.py", line 159, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/util/connection.py", line 80, in create_connection
    raise err
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/util/connection.py", line 70, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connectionpool.py", line 343, in _make_request
    self._validate_conn(conn)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn
    conn.connect()
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connection.py", line 301, in connect
    conn = self._new_conn()
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connection.py", line 168, in _new_conn
    self, "Failed to establish a new connection: %s" % e)
urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7fe3d1878c50>: Failed to establish a new connection: [Errno 101] Network is unreachable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/urllib3/util/retry.py", line 399, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /alan-nlp/resources/models-v0.4/NER-conll03--h256-l1-b32-p3-0.5-%2Bglove%2Bnews-forward-fast%2Bnews-backward-fast-normal-locked0.5-word0.05--release_4/en-ner-fast-conll03-v0.4.pt (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe3d1878c50>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xx/REL/rel_link.py", line 24, in <module>
    tagger_ner = SequenceTagger.load("ner-fast")
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/flair/nn.py", line 79, in load
    model_file = cls._fetch_model(str(model))
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/flair/models/sequence_tagger_model.py", line 1013, in _fetch_model
    model_name = cached_path(model_map[model_name], cache_dir=cache_dir)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/flair/file_utils.py", line 88, in cached_path
    return get_from_cache(url_or_filename, dataset_cache)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/flair/file_utils.py", line 160, in get_from_cache
    response = requests.head(url, headers={"User-Agent": "Flair"})
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/api.py", line 104, in head
    return request('head', url, **kwargs)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/api.py", line 61, in request
    return session.request(method=method, url=url, **kwargs)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/home/xx/miniconda3/envs/rel/lib/python3.7/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='s3.eu-central-1.amazonaws.com', port=443): Max retries exceeded with url: /alan-nlp/resources/models-v0.4/NER-conll03--h256-l1-b32-p3-0.5-%2Bglove%2Bnews-forward-fast%2Bnews-backward-fast-normal-locked0.5-word0.05--release_4/en-ner-fast-conll03-v0.4.pt (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7fe3d1878c50>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

However, it doesn't seem that my server has any network problems. Is there anything I can do to fix this?

"Illegal instruction (core dumped)" error, when I tried to do the "installation from source".

I did an installation from source but the "Illegal instruction (core dumped)" error occurred when I tried to run it.

This error occurs in PyTorch in Flair. If you use the latest version of PyTorch (1.6.0) and an old CPU that does not support AVX, then this error occurs. You can avoid this error by using PyTorch (1.5.1). For more details, see here.

Do you think it is better to specify the version of the PyTorch should be installed in the REL's readme document?

Details

If you call find_mentions like:

...
mentions_dataset, total_ment = self.mention_detection.find_mentions(processed, tagger_ner)

then you get

Illegal instruction (core dumped)

when you are using

the latest version of PyTorch (1.6.0) and
CPU that does not support AVX

Solution

Uninstall the PyTorch 1.6.0 and install PyTorch 1.5.1

Retrieve the Mention's wikipedia URL

Dear developer,

Thanks for sharing such a wonderful research to the public. I wonder is there any way to retrieve the predicted mention's wikipedia url after apply the ED model? Thanks!

Need help to reproduce results

Hi, I am trying to reproduce your results for your REL (2014) model on the GERBIL platform. For Table 1 (EL), I am using the A2KB setting with strong annotation match. For Table 2 (ED), I am using the D2KB setting. I followed the steps mentioned in the tutorial to configure the server and I just used the files provided in the repo without any further training. I am using AIDA B as the test set. The URLs for Table 1 and 2 are http://gerbil.aksw.org/gerbil/experiment?id=202010250000 and http://gerbil.aksw.org/gerbil/experiment?id=202010270001 resp.
Please see my results below.

	In KB Macro F1 (strong matching/A2KB)	In KB Micro F1 (strong matching/A2KB)	In KB Macro F1 (D2KB)	In KB Micro F1 (D2KB)
Wiki 2014 (reported)	81.3	83.3	85.5	86.6
Wiki 2014 (my result)	71.02	71.78	76.83	76.9

Please let me know if I am incorrect in any of the steps.
Thanks

Error in Google Colab

Hello,

Here is my code:

from REL.entity_disambiguation import EntityDisambiguation
base_url = "/content"
wiki_version = "wiki_2019"
config = {"mode": "eval","model_path": "ed-wiki-2019" # model alias}
ed_model = EntityDisambiguation(base_url, wiki_version, config)

I get the following error:
OperationalError: unable to open database file

Any idea?

Many thanks

multilingual?

hello,

Does REL can only be used for English text? Is it available for other language like french if I train it with a french wikipedia corpus?

Best,
Chen SUN

generic files

Hello, I hope to use this technology in Chinese, but I encountered a problem in the process. I don’t know how to generate crosswikis_p_e_m.txt in the generic directory. Could you please explain to me, Thanks!

Case sensitivity issue with embeddings used for ED model 2019

The wikipedia2vec embeddings used for pretrained ED model 2019 are only for lower case words unlike those used for ED model 2014.

However, the training seems to be done without taking that into account as a result all words with at least a single upper case letter default to the None or unknown embedding. Is this a bug?

Wouldn't it be better to go with case-sensitive embeddings instead?

Fine-tune REL for queries

As previously discussed, REL does not seem to perform well with queries due to e.g. words being mostly lowercase. To handle queries in a more optimal fashion, the following tasks are defined:

Re-train ED with a query-based dataset such as GERDAQ.
Retrain Flair with lowercase input.
Truecase the input.

Licensing issue unidecode

Received an email today stating that Unidecode was released under GNU General Public License, which means it is hard to use in industry (due to licensing issues). Other packages [0] have dealt with this problem by replacing it with a package called anyascii.

The package is called in the following files:
https://github.com/informagi/REL/blob/master/REL/utils.py
https://github.com/informagi/REL/blob/master/REL/generate_train_test.py (only as comments)

[0] wagtail/wagtail#6244

Reasoning behind Wikipedia2vec and modularity of model

I'm just wondering how Wikipedia2vec was arrived at. I know there are other KGE options out there (I was just presented with a list of TransE, TransR, RESCAL, DistMult, ComplEx, and RotatE).

It seems like swapping out the Wikipedia2vec should be possible but I'm wondering if that's something that's been considered as a possible feature and whether there any known limitations. In the case that a team member wants to use a different model, I'm thinking of how easy it will be to use.

Using Spacy NER

Hello! I already have extracted the named entities from my text using Spacy NER and have the original text / NER label (GPE, PERSON, LOC, etc). I was wondering if / how I could use this in the pipeline to speed up inference?

[Docker] Flair model URL changed + permission issues

Two things are keeping the docker images on Dockerhub from working at the moment;

The Flair model repository has changed location (https://github.com/flairNLP/flair/blob/093eba99b7c2fd0007c85e3848188eac5d92e7d3/flair/models/sequence_tagger_model.py#L973)
Permissions for files downloaded from REL fileserver are incorrect.

To fix:

Rebuilding the images should fix the issue.
chown appropriate folders.

Fixes incoming, will close this when done. See also #45

Efficiency issues

I am following your tutorial to integrate the end-to-end Entity Linking in my code to link mentions in a large number of documents. I have created a single instance of EntityDisambiguation class. But I find that the code runs slower as the iterations progress. I profiled the code to find that the bottleneck is in the method get_data_items() where the embeddings are being updated (self.__update_embeddings). Is there any way in which I can speed up the entity linking?

Add ED-only feature to the API

MD = Mention Detection, ED = Entity Disambiguation, EL = Entity Linking

About the data of Wikipedia

Thanks for your patient reply in email.
I downloaded the data of wikipedia2014 corpus, I have never seen Wikipedia data of this format, can you show me where to download Wikipedia of this format？To get p(e|m), I need this data, but I can not find Wikipedia data of this format in Wikipedia official website.
Thanks.

Mirroring results from public API

I setup a mirror of your API internally for use but the results I'm getting back don't look like the public API. I'm wondering if you could share your configuration for the public API.

Ill make a ticket before I leave, but just to give an example for the phrase:

"If you're going to try, go all the way - Charles Bukowski"

The public API returns:
[[41, 16, 'Charles Bukowski', 'Charles_Bukowski', 0.9880370497703552, 'PER', 0.0]]

The current output I'm seeing is:

[[24, 14, 'go all the way', 'Key_West', 0.8055737256826012, 28, '#NGRAM#'], [10, 8, 'going to', 'The_League_of_Extraordinary_Gentlemen', 0.7770389829061386, 32, '#NGRAM#'], [16, 7, 'to try,', 'Indian_National_Army_trials', 0.37921062453904336, 2, '#NGRAM#'], [41, 16, 'Charles Bukowski', 'Charles_Bukowski', 0.6841414231561657, 8420, '#NGRAM#'], [0, 2, 'If', 'If_(magazine)', 0.9085130370172767, 2580, '#NGRAM#'], [3, 6, "you're", 'Franklin_D._Roosevelt', 0.26132711208586534, 261, '#NGRAM#']]

Can you share how to get from the setup in the repo as-is to your results (which are obviously much better!)

AttributeError: 'GenTrainingTest' object has no attribute '_MentionDetection__sentences_doc'

Minimal reproduction example with REL:

# test_rel.py
from REL.generate_train_test import GenTrainingTest
from REl.wikipedia import Wikipedia

wikipedia = Wikipedia("/path/to/REL/", "wiki_2019")
data_handler = GenTrainingTest("/path/to/REL/", "wiki_2019", wikipedia)
data_handler.process_aida("train")

Loaded wiki disambiguation index
Loaded wiki redirects index
Loaded entity index
Trackback (most recent call last):
  File "test_rel.py", line 9, in <module>:
    data_handler.process_aida("train")
  File "/path/to/REL/REL/generate_train_test.py", line 331, in process_aida
    self.__save(self.__format(contents), "aida_train")
  File "/path/to/REL/REL/generate_train_test.py", line 51, in __format
    start, end, idx_sent, sentence
  File "/path/to/REL/REL/mention_detection.py", line 42, in _get_ctxt
    if idx_sent < len(self.__sentences_doc):
AttributeError: 'GenTrainingTest' object has no attribute '_MentionDetection__sentences_doc'

Minimal basic example of the same issue:

# test.py
class A:
    def test(self):
        print(self.__attribute)

class B(A):
    def _format(self):
        self.__attribute = "test"
        self.test()

Traceback (most recent call last):
  File "test.py", line 16, in <module>
    B()._format()
  File "test.py", line 13, in _format
    self.test()
  File "test.py", line 7, in test
    print(self.__attribute)
AttributeError: 'B' object has no attribute '_A__attribute'

After some investigation, this is due to a feature called name mangling that happens with double-underscore'd variables.

Any identifier of the form __spam (at least two leading underscores, at most one trailing underscore) is textually replaced with _classname__spam, where classname is the current class name with leading underscore(s) stripped.

This, in combination with the inheritance setup, breaks the code.

Simple fix: use single instead of double underscores.

Better fix: don't inherit from MentionDetection, instead create MentionDetection object in GenTrainingTest and pass data around. Inheriting from MentionDetection simply to use some overlapping functions (that communicate via object attributes) seems like a flawed design to me, so we might as well fix that instead of using a workaround IMO.

REL API format update

Remove spans parameter from EL functionality
Change output format to [start, length, mention, conf_ED, conf_NER, TYPE]

Two issues for my project

Hello, I am currently doing a project and I'd like to use REL but I am facing some problems.

The English preload models works well (wiki_2014, and 2019) but I need to evaluate the performance on the datasets by myself and I can't manage to do it even with the tutorials.
I also need to do it in French, so I've done the tutorial with the french wikipedia dump but I have problems with the anchor files wich are not the same format as in wiki_2014 for instance.

I am not that good in this domain so I'll be glad if someone of your team could help me!

Thank you !

Return valid JSON when no entities are found

At the moment, REL just throws an error internally and returns an empty page through the API when no entities are found. This requires the user to wrap JSON decoding in a try/except block which I believe to be undesirable.

Should be an easy fix and it will tidy up the interface a bit I think! I will do it myself :-)

what is tagger_ner used for?

The part II in the examples define the variable, loading flair fast-ner, but then the utility is never used again.

Is this correct? I'm trying to determine if it's needed at all.. It seems like it should be.

Problem as I setting up entity linking

As I followed up with tutorial to set up the package, it seems like there is a json file not reading properly. Thanks!

Integrate n-gram for mention detection

As the title states, integrate n-gram detection for mention detection. This is beneficial for high Recall cases.

TODO:

Base class implementation
Alter tutorials, where n-gram may be used as an alternative to Flair.

Details:

Implementation is a class similar to Flair.
It can be reused by users for a custom MD module.

Additionally
Users may use n-gram class as an example for creating their own MD module. Reduces complexity due to prediction on a per sentence basis. Before this update, the user would have to iterate over his/her sentences, give us their results, which would mean iterating twice per sentence. Using this method, the number of iterations per sentence is reduced to one. The function 'find_spans()' will solely be used for ED.

FutureWarning: The sklearn.linear_model.logistic module is deprecated in version 0.22 and will be removed in version 0.24.

I get the following warning when using REL:

FutureWarning: The sklearn.linear_model.logistic module is deprecated in version 0.22 and will be removed in version 0.24.
The corresponding classes / functions should instead be imported from sklearn.linear_model. Anything that cannot be imported from sklearn.linear_model is now part of the private API.
warnings.warn(message, FutureWarning)

This was after downgrading scikit-learn, because it does not work anymore in the newest version of scikit-learn.

Evaluation on EL

Hi, I am confused about the evaluation of EL.
In ED only, each mention has a correct candidate entity (annotated data), which we can calculate easily F1.
However, in EL, some mentions extracted by NER may not have a gold result (some mentions are not in the annotation data), or NER model may miss the correct mention existing in the annotation data.
How can I validate the performance of the EL model (How to calculate F1)?
Thanks~

Batching MD step

Documents have a varying number of sentences. To optimize this we may process all documents into sentences, where we store indexes to know which sentences belong to which documents. Then we may create equal-sized batches of these sentences, which can then be parsed for MD.

Implement GET request handling for status badge

Thinking about how to make the repository look nice, I came across Uptime Robot. It can send a GET request to the API every 5+ minutes, and we can use that result to put a API status badge in the repository readme:

(anything more fancy than a standard GET request is not free anymore... 😞 )

Would this be a nice addition? I could add it if people agree it looks nice. This would be separate from the CI build badge (#7).

Requirements for training on Arabic

Hi,
Is it possible to train the model on Arabic language ?
Is there any specific requirement ?
Thanks in advance,

Confidence measure

Implement Logistic Regression using the linear output scores of the ED network to obtain confidence scores.

ED:

Script to train LR.
Store model for offline usage.
Given LR model, add confidence score to output network.
Update current tutorials.

General

Update download files (2014 + 2019 corpus)

Continuous integration

To do:

Add .travis.yaml OR use Github workflow
Write some NER tests
Write some ED tests

Something to discuss: we cannot really expect certain model outputs, as they might change with updates to the code etc. We probably do not want to upload complete collections of necessary files (e.g. wiki folders, embeddings etc) for each CI run. Thus, I think we either have to mock some parts of the pipeline or accept reaaaally long CI runs. Thoughts?

Error while trying to do end-to-end EL

Hi REL folks,

I've been trying to follow the end-to-end EL tutorial, but I'm running into the following error:

Traceback (most recent call last):
  File "rel_test.py", line 2, in <module>
    from REL.mention_detection import MentionDetection
  File "/users/yh31/data/yh31/envs/entity_linking/lib/python3.7/site-packages/REL/mention_detection.py", line 5, in <module>
    from REL.mention_detection_base import MentionDetectionBase
  File "/users/yh31/data/yh31/envs/entity_linking/lib/python3.7/site-packages/REL/mention_detection_base.py", line 5, in <module>
    from REL.utils import modify_uppercase_phrase, split_in_words
  File "/users/yh31/data/yh31/envs/entity_linking/lib/python3.7/site-packages/REL/utils.py", line 13, in <module>
    def fetch_model(path_or_url, cache_dir=flair.cache_root / "models/taggers"):
TypeError: unsupported operand type(s) for /: 'str' and 'str'

The code I tried running, which is basically just the tutorial code with my own paths, is:


from REL.mention_detection import MentionDetection
from REL.utils import process_results
from REL.entity_disambiguation import EntityDisambiguation
from REL.ner import Cmns, load_flair_ner

wiki_version = "wiki_2019"
base_url = "/users/yh31/data/yh31/packages/REL_data"


def example_preprocessing():
    # user does some stuff, which results in the format below.
    text = "Obama will visit Germany. And have a meeting with Merkel tomorrow."
    processed = {"test_doc1": [text, []], "test_doc2": [text, []]}
    return processed

input_text = example_preprocessing()


mention_detection = MentionDetection(base_url, wiki_version)
tagger_ner = load_flair_ner("ner-fast")
tagger_ngram = Cmns(base_url, wiki_version, n=5)
mentions_dataset, n_mentions = mention_detection.find_mentions(input_text, tagger_ner)


config = {
    "mode": "eval",
    "model_path": "/users/yh31/data/yh31/packages/REL_data/ed-wiki-2019",
}

model = EntityDisambiguation(base_url, wiki_version, config)
predictions, timing = model.predict(mentions_dataset)
print(predictions)
print(timing)

result = process_results(mentions_dataset, predictions, input_text)
print(result)

Am I making a mistake with the paths or something? Should I be using path objects instead of strings?

Apologies for this neophyte question! Looking forward to using your software!

[Bug] Join paths the right way

Either use os.path.join or pathlib instead of + and .format to join paths.

That is, instead of this:

"{}/basic_data/wiki_disambiguation_pages.txt".format(self.base_url),

You could do this:

os.path.join(self.base_url, "/basic_data/wiki_disambiguation_pages.txt")

There are several bugs on the master branch ( specially on REL.wikipedia and REL.wikipedia_yago_freq ) because paths are not being joined in a consistent fashion. I suggest you try to run your own tutorials and then you'll see what I mean. They don't work unless I manually fix the way paths are joined.

CodeQL fails

Added CodeQL and it currently fails when parsing our Java code with the following error:

Error: We were unable to automatically build your code. Please replace the call to the autobuild action with your custom build steps. The process '/opt/hostedtoolcache/CodeQL/0.0.0-20200826/x64/codeql/java/tools/autobuild.sh' failed with exit code 1

Suggested fix is to ignore Java code in the check.

[Docker] error running the image

The log of running the image is as follow ( tried it with automatic and manual(5555) ports):

[#######:~]$ docker run -P --rm -it informagi/rel:2019 2020-08-25 03:18:22,260 https://s3.eu-central-1.amazonaws.com/alan-nlp/resources/models-v0.4/NER-conll03--h256-l1-b32-p3-0.5-%2Bglove%2Bnews-forward-fast%2Bnews-backward-fast-normal-locked0.5-word0.05--release_4/en-ner-fast-conll03-v0.4.pt not found in cache, downloading to /tmp/tmphlgzrwlk 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 256774339/256774339 [00:12<00:00, 20665722.77B/s] 2020-08-25 03:18:35,239 copying /tmp/tmphlgzrwlk to cache at /root/.flair/models/en-ner-fast-conll03-v0.4.pt 2020-08-25 03:18:35,458 removing temp file /tmp/tmphlgzrwlk 2020-08-25 03:18:35,493 loading file /root/.flair/models/en-ner-fast-conll03-v0.4.pt 2020-08-25 03:18:37,523 http://gem.cs.ru.nl/ed-wiki-2019.tar.gz not found in cache, downloading to /tmp/tmpeqim6x5s 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 1019959/1019959 [00:00<00:00, 1610233.38B/s] 2020-08-25 03:18:38,400 copying /tmp/tmpeqim6x5s to cache at /root/.rel_cache/ed-wiki-2019.tar.gz 2020-08-25 03:18:38,403 removing temp file /tmp/tmpeqim6x5s Traceback (most recent call last): File "/opt/conda/lib/python3.7/runpy.py", line 193, in _run_module_as_main "__main__", mod_spec) File "/opt/conda/lib/python3.7/runpy.py", line 85, in _run_code exec(code, run_globals) File "/workspace/REL/REL/server.py", line 163, in <module> args.base_url, args.wiki_version, {"mode": "eval", "model_path": args.ed_model} File "/workspace/REL/REL/entity_disambiguation.py", line 47, in __init__ "entity_word_embedding", "{}/{}/generated/".format(base_url, wiki_version), File "/workspace/REL/REL/db/generic.py", line 35, in __init__ self.db = self.initialize_db(path_db, table_name, columns) File "/workspace/REL/REL/db/base.py", line 41, in initialize_db db = sqlite3.connect(fname, isolation_level=None) sqlite3.OperationalError: unable to open database file

Extend model (NER, ED) loading functionality for custom models using aliases

We would like to implement something similar to what Flair does, i.e.:

model = SequenceTagger.load("alias-for-model")

where alias-for-model could also point to one of our custom trained models, e.g. CoNLL03 with lowercased entities. Some useful functions would be https://github.com/flairNLP/flair/blob/f5d846ffcaf56d9f90cde484b234685c17a23096/flair/models/sequence_tagger_model.py#L865 and https://github.com/flairNLP/flair/blob/f5d846ffcaf56d9f90cde484b234685c17a23096/flair/file_utils.py#L75, that way we barely have to implement anything 🤓

About evaluate on GERBIL

Hi, I am trying to reproduce the results on the GERBIL platform.
On my remote server, I start gerbil and receive Starting Coyote HTTP/1.1 on http-1235.
I also start the API,and set server_address = ("localhost", 5555).
Finally, I configure an experiment. Where the URI field be filled with:
http://remote_address:1235/gerbil-spotWrapNifWS4Test/myalgorithm.
When I click on the Add Annotator button, I get the error

Warning! There was an error while testing the annotator.
Exception while sending request.

what should I do?
Thanks~

Properly handle unsupported requests

Implement responses for e.g. HEAD requests and improperly formatted POST requests.

REL becomes unresponsive at seemingly random intervals

I can't really reproduce this as of yet. After being online for a while, REL starts to just timeout on requests without crashing. See below for the systemd journal for REL which does not really help figuring out what is going wrong...

I'm going to try and figure this out, any idea's are welcome.

[...truncated...]
Apr 29 06:50:13 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: 91.218.85.100 - - [28/Apr/2020 22:36:58] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 5.55.107.197 - - [28/Apr/2020 22:56:55] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 94.140.114.17 - - [28/Apr/2020 23:00:50] "GET / HTTP/1.0" 200 -
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: Exception happened during processing of request from ('96.69.158.193', 60760)
Apr 29 06:50:13 gem bash[3234]: Traceback (most recent call last):
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 06:50:13 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 06:50:13 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 06:50:13 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 06:50:13 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 06:50:13 gem bash[3234]:     self.handle()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 06:50:13 gem bash[3234]:     self.handle_one_request()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 06:50:13 gem bash[3234]:     method()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 57, in do_POST
Apr 29 06:50:13 gem bash[3234]:     content_length = int(self.headers["Content-Length"])
Apr 29 06:50:13 gem bash[3234]: TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: 61.219.11.153 - - [28/Apr/2020 23:08:57] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 185.153.197.10 - - [28/Apr/2020 23:25:48] code 400, message Bad HTTP/0.9 request type ('\x03\x00\x00/*à\x00\x00\x00\x00\x00Cookie:')
Apr 29 06:50:13 gem bash[3234]: [44B blob data]
Apr 29 06:50:13 gem bash[3234]: /*à
Apr 29 06:50:13 gem bash[3234]: Cookie: mstshash=Administr" 400 -
Apr 29 06:50:13 gem bash[3234]: 185.153.197.10 - - [28/Apr/2020 23:28:36] code 400, message Bad HTTP/0.9 request type ('\x03\x00\x00/*à\x00\x00\x00\x00\x00Cookie:')
Apr 29 06:50:13 gem bash[3234]: [44B blob data]
Apr 29 06:50:13 gem bash[3234]: /*à
Apr 29 06:50:13 gem bash[3234]: Cookie: mstshash=Administr" 400 -
Apr 29 06:50:13 gem bash[3234]: 91.5.90.227 - - [28/Apr/2020 23:34:06] "GET / HTTP/1.0" 200 -
Apr 29 06:50:13 gem bash[3234]: 80.224.175.243 - - [29/Apr/2020 00:27:54] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 114.67.202.10 - - [29/Apr/2020 02:39:50] "GET /TP/public/index.php HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 114.67.202.10 - - [29/Apr/2020 02:39:51] "GET /TP/public/index.php?s=index/\think\app/invokefunction&function=call_user_func_array&vars[0]=phpinfo&vars[1][]=1 HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 114.67.202.10 - - [29/Apr/2020 02:39:51] "POST /TP/public/index.php?s=captcha HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: Exception happened during processing of request from ('114.67.202.10', 33748)
Apr 29 06:50:13 gem bash[3234]: Traceback (most recent call last):
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 06:50:13 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 06:50:13 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 06:50:13 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 06:50:13 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 06:50:13 gem bash[3234]:     self.handle()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 06:50:13 gem bash[3234]:     self.handle_one_request()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 06:50:13 gem bash[3234]:     method()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 06:50:13 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 06:50:13 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 06:50:13 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 06:50:13 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 06:50:13 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 06:50:13 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: 114.67.202.10 - - [29/Apr/2020 02:39:52] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 83.97.20.21 - - [29/Apr/2020 02:54:46] "GET / HTTP/1.0" 200 -
Apr 29 06:50:13 gem bash[3234]: 162.243.139.140 - - [29/Apr/2020 03:37:23] "GET /portal/redlion HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 92.63.194.30 - - [29/Apr/2020 04:33:17] code 400, message Bad HTTP/0.9 request type ('\x03\x00\x00/*à\x00\x00\x00\x00\x00Cookie:')
Apr 29 06:50:13 gem bash[3234]: [42B blob data]
Apr 29 06:50:13 gem bash[3234]: /*à
Apr 29 06:50:13 gem bash[3234]: Cookie: mstshash=Administr" 400 -
Apr 29 06:50:13 gem bash[3234]: 200.57.192.246 - - [29/Apr/2020 04:54:23] "POST /cgi-bin/mainfunction.cgi HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: Exception happened during processing of request from ('200.57.192.246', 59743)
Apr 29 06:50:13 gem bash[3234]: Traceback (most recent call last):
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 06:50:13 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 06:50:13 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 06:50:13 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 06:50:13 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 06:50:13 gem bash[3234]:     self.handle()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 06:50:13 gem bash[3234]:     self.handle_one_request()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 06:50:13 gem bash[3234]:     method()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 06:50:13 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 06:50:13 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 06:50:13 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 06:50:13 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 06:50:13 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 06:50:13 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: 193.92.240.102 - - [29/Apr/2020 05:55:45] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 179.99.193.177 - - [29/Apr/2020 06:05:02] "GET / HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: 192.186.94.70 - - [29/Apr/2020 06:50:13] "POST /cgi-bin/mainfunction.cgi HTTP/1.1" 200 -
Apr 29 06:50:13 gem bash[3234]: ----------------------------------------
Apr 29 06:50:13 gem bash[3234]: Exception happened during processing of request from ('192.186.94.70', 54671)
Apr 29 06:50:13 gem bash[3234]: Traceback (most recent call last):
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 06:50:13 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 06:50:13 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 06:50:13 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 06:50:13 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 06:50:13 gem bash[3234]:     self.handle()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 06:50:13 gem bash[3234]:     self.handle_one_request()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 06:50:13 gem bash[3234]:     method()
Apr 29 06:50:13 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 06:50:13 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 10:46:28 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 10:46:28 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 10:46:28 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 10:46:28 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 10:46:28 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: 71.6.232.4 - - [29/Apr/2020 07:03:04] "GET / HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 203.170.153.129 - - [29/Apr/2020 07:08:31] "GET / HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 187.190.221.119 - - [29/Apr/2020 07:25:36] "POST /cgi-bin/mainfunction.cgi HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: Exception happened during processing of request from ('187.190.221.119', 21394)
Apr 29 10:46:28 gem bash[3234]: Traceback (most recent call last):
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 10:46:28 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 10:46:28 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 10:46:28 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 10:46:28 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 10:46:28 gem bash[3234]:     self.handle()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 10:46:28 gem bash[3234]:     self.handle_one_request()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 10:46:28 gem bash[3234]:     method()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 10:46:28 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 10:46:28 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 10:46:28 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 10:46:28 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 10:46:28 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 10:46:28 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 08:10:25] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: Exception happened during processing of request from ('120.77.224.17', 43540)
Apr 29 10:46:28 gem bash[3234]: Traceback (most recent call last):
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 10:46:28 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 10:46:28 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 10:46:28 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 10:46:28 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 10:46:28 gem bash[3234]:     self.handle()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 10:46:28 gem bash[3234]:     self.handle_one_request()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 394, in handle_one_request
Apr 29 10:46:28 gem bash[3234]:     self.raw_requestline = self.rfile.readline(65537)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socket.py", line 589, in readinto
Apr 29 10:46:28 gem bash[3234]:     return self._sock.recv_into(b)
Apr 29 10:46:28 gem bash[3234]: ConnectionResetError: [Errno 104] Connection reset by peer
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: 196.52.43.102 - - [29/Apr/2020 08:25:25] "GET / HTTP/1.0" 200 -
Apr 29 10:46:28 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 08:29:59] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 185.202.2.147 - - [29/Apr/2020 08:33:37] code 400, message Bad HTTP/0.9 request type ('\x03\x00\x00/*à\x00\x00\x00\x00\x00Cookie:')
Apr 29 10:46:28 gem bash[3234]: [43B blob data]
Apr 29 10:46:28 gem bash[3234]: /*à
Apr 29 10:46:28 gem bash[3234]: Cookie: mstshash=Administr" 400 -
Apr 29 10:46:28 gem bash[3234]: 193.118.53.210 - - [29/Apr/2020 08:38:00] "GET / HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 35.133.57.185 - - [29/Apr/2020 08:48:50] "GET / HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 58.156.198.64 - - [29/Apr/2020 08:51:38] "POST /api:80 HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 80.82.78.104 - - [29/Apr/2020 08:54:05] "POST /cgi-bin/mainfunction.cgi HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: Exception happened during processing of request from ('80.82.78.104', 44046)
Apr 29 10:46:28 gem bash[3234]: Traceback (most recent call last):
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 10:46:28 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 10:46:28 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 10:46:28 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 10:46:28 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 10:46:28 gem bash[3234]:     self.handle()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 10:46:28 gem bash[3234]:     self.handle_one_request()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 10:46:28 gem bash[3234]:     method()
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 10:46:28 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 10:46:28 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 10:46:28 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 10:46:28 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 10:46:28 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 10:46:28 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 10:46:28 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 10:46:28 gem bash[3234]: ----------------------------------------
Apr 29 10:46:28 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 09:12:33] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 196.52.43.104 - - [29/Apr/2020 09:19:57] "GET / HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 10:12:57] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 10:24:25] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:25:40] "GET /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:25:41] "GET /favicon.ico HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:30:17] "POST /api:80 HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:30:36] code 501, message Unsupported method ('Post')
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:30:36] "Post /api:80 HTTP/1.1" 501 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:30:51] "POST /api:80 HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:31:48] "POST /api:80 HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:37:51] "POST /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:41:06] "POST /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:42:28] "POST /api:80 HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:45:39] "POST /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:27] "POST /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:28] "POST /api HTTP/1.1" 200 -
Apr 29 10:46:28 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:28] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:29] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:30] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:31] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:32] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:33] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:34] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:35] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:36] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:37] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:37] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:38] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:39] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:40] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:41] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:42] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:43] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:44] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:45] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:46] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:47] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:48] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:49] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:50] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:51] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:52] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:53] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:54] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:55] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:55] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:56] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:57] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:58] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:46:59] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:00] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:01] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:02] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:03 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:03] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:03] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:04] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:05] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:06] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:07] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:08] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:09] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:10] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:10] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:11] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:12] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:13] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:14] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:15] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:16] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:17] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:17] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:18] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:19] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:20] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:21] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:22] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:23] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:24] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:24] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:25] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:26] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:27] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:28] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:29] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:30] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:31] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:32] "POST /api HTTP/1.1" 200 -
Apr 29 10:47:32 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:32] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:32] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:33] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:34] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:35] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:36] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:37] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:38] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:39] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:40] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:40] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:41] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:42] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:43] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:44] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:45] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:46] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:47] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:47] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:48] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:49] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:50] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:51] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:52] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:53] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:54] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:55] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:56] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:56] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:57] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:58] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:47:59] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:00] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:01] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:02] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:03] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:04] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:04 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:04] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:04] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:05] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:06] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:07] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:08] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:09] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:10] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:11] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:12] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:12] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:13] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:14] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:15] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:16] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:17] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:18] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:19] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:20] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:21] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:22] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:23] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:24] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:25] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:26] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:27] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:28] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:29] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:29] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:30] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:31] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:32] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:33] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:34] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:35] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:36] "POST /api HTTP/1.1" 200 -
Apr 29 10:48:36 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:36] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:36] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:37] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:38] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:39] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:40] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:41] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:42] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:43] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:44] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:45] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:45] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:46] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:47] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:48] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:49] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:50] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 10:48:51] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 11:33:43] "GET /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:27] "GET / HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:27] "GET /home.asp HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:28] "GET /login.cgi?uri= HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:28] "GET /vpn/index.html HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:29] "GET /cgi-bin/luci HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:29] "GET /dana-na/auth/url_default/welcome.cgi HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:30] "GET /remote/login?lang=en HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:30] "GET /index.asp HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 140.238.42.16 - - [29/Apr/2020 11:36:31] "GET /htmlV/welcomeMain.htm HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 11:45:34] "GET /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 5.79.193.51 - - [29/Apr/2020 12:40:19] "GET / HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 193.168.152.254 - - [29/Apr/2020 12:45:26] "GET / HTTP/1.0" 200 -
Apr 29 13:10:58 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 12:51:10] "GET /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:45] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:46] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:47] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:48] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:49] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:50] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:51] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:52] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:53] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:54] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:55] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:56] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:57] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:58] "POST /api HTTP/1.1" 200 -
Apr 29 13:10:58 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:58] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:10:59] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:00] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:01] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:02] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:03] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:04] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:05] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:06] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:07] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:08] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:08] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:09] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:10] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:11] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:12] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:13] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:14] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:15] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:16] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:17] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:18] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:19] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:20] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:21] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:22] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:23] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:24] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:24] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:25] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:26] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:27] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:28] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:29] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:30] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:31] "POST /api HTTP/1.1" 200 -
Apr 29 13:11:31 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:31] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:32] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:33] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:34] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:35] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:36] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:37] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:38] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:39] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:39] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:40] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:41] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:42] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:43] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:44] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:45] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:46] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:46] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:47] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:48] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:49] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:50] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:51] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:52] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:53] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:54] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:55] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:56] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:57] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:58] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:11:59] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:12:00] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:12:18] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:12:19] "POST /api HTTP/1.1" 200 -
Apr 29 13:12:20 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:12:20] "POST /api HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 77.22.253.174 - - [29/Apr/2020 13:12:20] "POST /api HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 80.82.70.118 - - [29/Apr/2020 14:48:37] "GET / HTTP/1.0" 200 -
Apr 29 23:30:56 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 14:50:37] "GET /api HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 162.243.144.226 - - [29/Apr/2020 15:00:36] "GET /hudson HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 95.90.53.102 - - [29/Apr/2020 15:52:42] "GET / HTTP/1.0" 200 -
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: Exception happened during processing of request from ('45.88.148.162', 58881)
Apr 29 23:30:56 gem bash[3234]: Traceback (most recent call last):
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 23:30:56 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 23:30:56 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 23:30:56 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 23:30:56 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 23:30:56 gem bash[3234]:     self.handle()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 23:30:56 gem bash[3234]:     self.handle_one_request()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 394, in handle_one_request
Apr 29 23:30:56 gem bash[3234]:     self.raw_requestline = self.rfile.readline(65537)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socket.py", line 589, in readinto
Apr 29 23:30:56 gem bash[3234]:     return self._sock.recv_into(b)
Apr 29 23:30:56 gem bash[3234]: ConnectionResetError: [Errno 104] Connection reset by peer
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: 45.88.148.162 - - [29/Apr/2020 16:00:01] code 501, message Unsupported method ('HEAD')
Apr 29 23:30:56 gem bash[3234]: 45.88.148.162 - - [29/Apr/2020 16:00:01] "HEAD /robots.txt HTTP/1.0" 501 -
Apr 29 23:30:56 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 16:05:16] "GET /api HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 131.174.30.199 - - [29/Apr/2020 16:15:48] "GET /api HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 31.192.150.141 - - [29/Apr/2020 16:32:10] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 134.169.109.83 - - [29/Apr/2020 16:57:03] "GET / HTTP/1.0" 200 -
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: Exception happened during processing of request from ('45.88.148.162', 62235)
Apr 29 23:30:56 gem bash[3234]: Traceback (most recent call last):
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 23:30:56 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 23:30:56 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 23:30:56 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 23:30:56 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 23:30:56 gem bash[3234]:     self.handle()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 23:30:56 gem bash[3234]:     self.handle_one_request()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 394, in handle_one_request
Apr 29 23:30:56 gem bash[3234]:     self.raw_requestline = self.rfile.readline(65537)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socket.py", line 589, in readinto
Apr 29 23:30:56 gem bash[3234]:     return self._sock.recv_into(b)
Apr 29 23:30:56 gem bash[3234]: ConnectionResetError: [Errno 104] Connection reset by peer
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: 45.88.148.162 - - [29/Apr/2020 17:22:39] code 501, message Unsupported method ('HEAD')
Apr 29 23:30:56 gem bash[3234]: 45.88.148.162 - - [29/Apr/2020 17:22:39] "HEAD /robots.txt HTTP/1.0" 501 -
Apr 29 23:30:56 gem bash[3234]: 132.248.193.80 - - [29/Apr/2020 17:39:30] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 162.243.136.8 - - [29/Apr/2020 18:06:05] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 36.91.177.167 - - [29/Apr/2020 19:04:42] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: Exception happened during processing of request from ('177.137.97.38', 44279)
Apr 29 23:30:56 gem bash[3234]: Traceback (most recent call last):
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 23:30:56 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 23:30:56 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 23:30:56 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 23:30:56 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 23:30:56 gem bash[3234]:     self.handle()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 23:30:56 gem bash[3234]:     self.handle_one_request()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 23:30:56 gem bash[3234]:     method()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 57, in do_POST
Apr 29 23:30:56 gem bash[3234]:     content_length = int(self.headers["Content-Length"])
Apr 29 23:30:56 gem bash[3234]: TypeError: int() argument must be a string, a bytes-like object or a number, not 'NoneType'
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:27] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:28] "GET /home.asp HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:28] "GET /login.cgi?uri= HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:29] "GET /vpn/index.html HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:30] "GET /cgi-bin/luci HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:31] "GET /dana-na/auth/url_default/welcome.cgi HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:31] "GET /remote/login?lang=en HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:32] "GET /index.asp HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 104.225.218.14 - - [29/Apr/2020 20:50:32] "GET /htmlV/welcomeMain.htm HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 187.120.75.83 - - [29/Apr/2020 23:25:30] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 96.47.122.230 - - [29/Apr/2020 23:25:45] "POST /cgi-bin/mainfunction.cgi HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: Exception happened during processing of request from ('96.47.122.230', 43367)
Apr 29 23:30:56 gem bash[3234]: Traceback (most recent call last):
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
Apr 29 23:30:56 gem bash[3234]:     self.process_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 347, in process_request
Apr 29 23:30:56 gem bash[3234]:     self.finish_request(request, client_address)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 360, in finish_request
Apr 29 23:30:56 gem bash[3234]:     self.RequestHandlerClass(request, client_address, self)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 31, in __init__
Apr 29 23:30:56 gem bash[3234]:     super().__init__(*args, **kwargs)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/socketserver.py", line 720, in __init__
Apr 29 23:30:56 gem bash[3234]:     self.handle()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 426, in handle
Apr 29 23:30:56 gem bash[3234]:     self.handle_one_request()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/http/server.py", line 414, in handle_one_request
Apr 29 23:30:56 gem bash[3234]:     method()
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 62, in do_POST
Apr 29 23:30:56 gem bash[3234]:     text, spans = self.read_json(post_data)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/REL/REL/server.py", line 76, in read_json
Apr 29 23:30:56 gem bash[3234]:     data = json.loads(post_data.decode("utf-8"))
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/__init__.py", line 348, in loads
Apr 29 23:30:56 gem bash[3234]:     return _default_decoder.decode(s)
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 337, in decode
Apr 29 23:30:56 gem bash[3234]:     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
Apr 29 23:30:56 gem bash[3234]:   File "/home/kdercksen/.conda/envs/rel/lib/python3.7/json/decoder.py", line 355, in raw_decode
Apr 29 23:30:56 gem bash[3234]:     raise JSONDecodeError("Expecting value", s, err.value) from None
Apr 29 23:30:56 gem bash[3234]: json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Apr 29 23:30:56 gem bash[3234]: ----------------------------------------
Apr 29 23:30:56 gem bash[3234]: 114.242.9.143 - - [29/Apr/2020 23:30:55] "GET / HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 114.242.9.143 - - [29/Apr/2020 23:30:55] "GET /home.asp HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 114.242.9.143 - - [29/Apr/2020 23:30:56] "GET /login.cgi?uri= HTTP/1.1" 200 -
Apr 29 23:30:56 gem bash[3234]: 114.242.9.143 - - [29/Apr/2020 23:30:56] "GET /vpn/index.html HTTP/1.1" 200 -
Apr 30 14:37:09 gem systemd[1]: Stopping Entity Linking service...
Apr 30 14:37:09 gem systemd[1]: Stopped Entity Linking service.
Apr 30 14:37:09 gem systemd[1]: Started Entity Linking service.
Apr 30 14:37:12 gem bash[11567]: No LR model found, confidence scores ED will be set to zero.
Apr 30 14:37:12 gem bash[11567]: Loading model from given path: /store/REL//wiki_2019/generated/model
Apr 30 14:37:12 gem bash[11567]: 2020-04-30 14:37:12,848 loading file /root/.flair/models/en-ner-fast-conll03-v0.4.pt
May 01 17:07:03 gem systemd[1]: Stopping Entity Linking service...
May 01 17:07:03 gem systemd[1]: Stopped Entity Linking service.
May 01 17:07:03 gem systemd[1]: Started Entity Linking service.
May 01 17:07:06 gem bash[3608]: No LR model found, confidence scores ED will be set to zero.
May 01 17:07:06 gem bash[3608]: Loading model from given path: /store/REL//wiki_2019/generated/model
May 01 17:07:06 gem bash[3608]: 2020-05-01 17:07:06,542 loading file /root/.flair/models/en-ner-fast-conll03-v0.4.pt

Error for start position of mentions

Issue

Returning the same start positions for different mentions

Examples

Example 1

In this example, mention “[128, 5, ‘Paris’...“, which the second arrow indicates, has a wrong start position. (The true start position is 449.)

IP_ADDRESS = "https://rel.cs.ru.nl/api"

def el(text):
    document = {
        "text": text,
        "spans": []
    }
    API_result = requests.post("{}".format(IP_ADDRESS), json=document).json()
    return API_result

# Input text
dialogue =  "Hi. I need to book a vacation to Long Beach between August 25 and September 3. Departure is from Paris\tWould you like to depart Paris on September 6th?\tPreferably by the 3rd. Is there anything available?\tSorry, looks like there is nothing available from Paris to Long Beach on September 3rd.\tI'm not quite sure I understand, is there anything available leaving Long Beach to go to Paris between August 25 and September 3rd?\tWould you like to depart Paris on September 6th?\tNo. I would like to leave Long Beach around the 25th of August to go to Paris for some reason. What is so confusing about that!?\tYou can leave Long Beach, USA and go to Paris, France on Tuesday, August 30th. Will I book this?\tFinally! No, don't book yet, I would like to know more about the hotel. Is there free breakfast?\tThere is free wifi.\tLook buddy, is there free breakfast or not? Tell me, am I gonna get eggs, toast, cereal, etc? You know? The good stuff?\tThere is free wifi at the hotel. \tWhat is the price of this toastless package?\tThis package costs 2191.55USD.\tIs this the cheapest option?\tYes, this is the cheapest option from Long Beach to Paris.\tAnd the hotel has how many stars?\tMuse Hotel has 2.0 stars.\tOk I will book this one.\tGreat. You will leave Paris on September 7"

# Entity Linking
el(dialogue)

# Output
#[
#    [33, 10, 'Long Beach', 'Long_Beach,_California', 0.37835263573950106, 0.9858004450798035, 'LOC'], 
#    [97, 5, 'Paris', 'Paris', 0.8168372931596612, 0.9900407195091248, 'LOC'], 
#    ---> [128, 5, 'Paris', 'Paris', 0.8369923447610185, 0.9948850274085999, 'LOC'], 
#    [254, 5, 'Paris', 'Paris', 0.7787875371965208, 0.9929811954498291, 'LOC'], 
#    ...
#    [381, 5, 'Paris', 'Paris', 0.8693552725957252, 0.9956340193748474, 'LOC'], 
#    ---> [128, 5, 'Paris', 'Paris', 0.9030778907304463, 0.9946692585945129, 'LOC'], 
#    [499, 10, 'Long Beach', 'Long_Beach,_California', 0.38097207439050973, 0.9697867035865784, 'LOC'], 
#    [545, 5, 'Paris', 'Paris', 0.8518866064471391, 0.9992392063140869, 'LOC'], 
#    ...
#]

Example 2

dialogue = 'Paris. Paris. Paris. Paris.'
el(dialogue)

# Output
# [[0, 5, 'Paris', 'Paris', 0.871144498463729, 0.9808972477912903, 'LOC'],
# [0, 5, 'Paris', 'Paris', 0.871144498463729, 0.9808972477912903, 'LOC'],
# [0, 5, 'Paris', 'Paris', 0.871144498463729, 0.9808972477912903, 'LOC'],
# [0, 5, 'Paris', 'Paris', 0.871144498463729, 0.9808972477912903, 'LOC']]

Cause

The pice of code in [1] is meant to translate local (within single sentence) span indices to global (in the whole piece of text) indices. However, if the sentences (in this case "Paris.", "Paris.", etc...) are identical, this will only find the first one and hence the offsets are not correct.
Also, the same issue can be found in [2].

Update 14-10-2020

How issues [1] and [2] affect the results.
See [3].

If input spans' length is len(spans) == 0, then include_offset become True and issue [1] affects the result.
If input spans' length is len(spans) > 0, then include_offset become False and issue [2] affects the result.
- Because start_pos become the same as ment["pos"], which is the result of [2]. (Maybe)

How to fix

Idea

Use
pos_start = text[pos_start:].find(sent) + pos_start
instead of using
pos_start = text.find(sent)

E.g., Regarding [2]:

...
i = 0
pos_start = 0 # Added
for sent in sentences:
    if len(sent.strip()) == 0:
        continue
    # Match gt to sentence.
    #pos_start = text.find(sent) # Original
    pos_start = text[pos_start:].find(sent) + pos_start # Added
    ...

I believe this correction also works for [1], I did not check it yet though.

Todo

Decide how to fix both issues
Fix the first issue mentioned in "cause" section
Fix the second issue

References

[1] https://github.com/informagi/REL/blob/master/REL/utils.py#L92
[2] https://github.com/informagi/REL/blob/master/REL/mention_detection.py#L87
[3] https://github.com/informagi/REL/blob/master/REL/utils.py#L94

404

I keep getting 404 error using the sample request from the README

ED model setup in the docker container

I am setting up a local instance of Radboud Entity Linker using a pre-build docker image using the following instructions.

docker run 
    -p 5555:5555 
    -v /path/to/generic:/workspace/generic 
    -v /path/to/wiki_2019:/workspace/wiki_2019 
    --rm -it informagi/rel
    python -m REL.server --bind 0.0.0.0 --port 5555 /workspace wiki_2019

I wonder how can I configure it to use the downloaded ED model ed-wiki-2019.tar.gz? Do I have to put it in a specific page or pass it to the server?

Possible to get back the Wikipedia2vec vectors from self-serve API?

I'm considering standing up an API but was wondering if I could stand it up to get back the trained vectors, particularly if I fine-tune vectors.

So for example with this request:

import requests

IP_ADDRESS = "http://localhost"
PORT = "1235"
text_doc = "If you're going to try, go all the way - Charles Bukowski"

document = {
    "text": text_doc,
    "spans": [],
}

API_result = requests.post("{}:{}".format(IP_ADDRESS, PORT), json=document).json()

I'd get back not just the linked entity but the Wiki2vec embedding vector of that linked entity.

Is it much work to set it up this way?

ner_wrapper malfunctions because of Flair transition to HuggingFace model hub

Loading models with our custom keys in REL/models/models.json does not work anymore because of new model loading functionality in Flair. E.g. running python -m REL.server ... --ner-model ner-fast-with-lowercase throws an error.

Traceback (most recent call last):
  File "/home/koen/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)                                                                                                                
  File "/home/koen/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 85, in _run_code                                                                        
    exec(code, run_globals)                                                                                                                                    
  File "/home/koen/projects/REL/REL/server.py", line 160, in <module>                                                                                          
    ner_model = load_flair_ner(args.ner_model)                                                                                                                 
  File "/home/koen/projects/REL/REL/ner/flair_wrapper.py", line 9, in load_flair_ner                                                                           
    return SequenceTagger.load(path_or_url)                                                                                                                    
  File "/home/koen/.pyenv/versions/text-processing/lib/python3.7/site-packages/flair/nn.py", line 81, in load                                                  
    model_file = cls._fetch_model(str(model))                                                                                                                  
  File "/home/koen/.pyenv/versions/text-processing/lib/python3.7/site-packages/flair/models/sequence_tagger_model.py", line 1232, in _fetch_model              
    return model_path                                                                                                                                          
UnboundLocalError: local variable 'model_path' referenced before assignment

Recommend Projects

React

A declarative, efficient, and flexible JavaScript library for building user interfaces.
Vue.js

🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript

TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
TensorFlow

An Open Source Machine Learning Framework for Everyone
Django

The Web framework for perfectionists with deadlines.
Laravel

A PHP framework for web artisans
D3

Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

javascript

JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
web

Some thing interesting about web. New door for the world.
server

A server is a program made to process requests and deliver data to clients.
Machine learning

Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Visualization

Some thing interesting about visualization, use data art
Game

Some thing interesting about game, make everyone happy.

Recommend Org

Facebook

We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft

Open source projects and samples from Microsoft.
Google

Google ❤️ Open Source for everyone.
Alibaba

Alibaba Open Source for everyone
D3

Data-Driven Documents codes.
Tencent

China tencent open source team.

informagi / rel Goto Github PK

rel's Introduction

REL: Radboud Entity Linker

Calling our API

Using REL as a Python package

Installation

Option 1: Installation using pip

Option 2: Installation using Docker

Prebuilt images

Build your own Docker image

Option 3: Installation from source code

Download data

Tutorials

REL variants

Efficiency of REL

Development

Cite

Contact

Acknowledgements

rel's People

Contributors

Stargazers

Watchers

Forkers

rel's Issues

Details

Solution

Issue

Examples

Example 1

Example 2

Cause

Update 14-10-2020

How to fix

Idea

Todo

References

Recommend Projects

Recommend Topics

Recommend Org