siscale / covid-19-elk Goto Github PK

Hello,

Right now there is a bug in the Kibana visualization covid 19 confirmed by country total tvsb. The US is not correctly represented.

In your post your dashboard seems to represent correctly the results.

Hi Siscale Team --

I'm using a vanilla, RPM-based distribution of Elastic Stack, and seeing the following error when starting Logstash with the logstash-github-covid-19-daily-reports-template.conf config loaded:

[2020-05-24T16:36:50,741][ERROR][logstash.filters.ruby ][main] Ruby exception occurred: A JSON text must at least contain two octets!

I've updated the relevant parts of the config as it suggests. Here are the changed bits:

@github_stored_hashes_path = "/tmp/covid-19-hashes.json" 
hosts => "my.elastic.host:9200" 
user => "${LOGSTASH_USERNAME}" (in /etc/default/logstash)
password => "${LOGSTASH_PASSWORD}" (in /etc/default/logstash)

The hashes file is chown'ed logstash:logstash and chmod'ed 644. Most of the time, when I start Logstash, I see just one document created:

{
  "_index": "covid-19-live-update",
  "_type": "_doc",
  "_id": "d7yER3IBTwALIWCBBH4E",
  "_version": 1,
  "_score": null,
  "_source": {
    "tags": [
      "check_github",
      "_rubyexception"
    ],
    "message": "first-time-run",
    "@version": "1",
    "@timestamp": "2020-05-24T16:28:52.700Z",
    "host": "my.server.com",
    "sequence": 0
  },
  "fields": {
    "@timestamp": [
      "2020-05-24T16:28:52.700Z"
    ]
  },
  "sort": [
    1590337732700
  ]
}

Once, when starting Logstash, I see something different than the above. http.poller.* fields all had content, and the message field itself contained HTML (appearing to be the actual HTML from the JHU CSV repo). Though, this document also has _rubyexception tag.

The last thing I can think to mention is that I initially missed loading the index template prior to starting Logstash. I am not certain that really matters. I completely deleted the index created with the single document, delete the index pattern, and started from the top. Got the same result.

Here's the basics of my environment.

• CentOS 7.7
• OpenJDK 1.8.0_242
• Elastic Stack 7.6.2

The system itself is somewhat anorexic (4GB RAM), but not running any production workloads otherwise.

Thanks for your help. The work is beautiful!