apachebeat sending wrong field type (string instead of numeric) about apachebeat HOT 15 CLOSED

@borian I'll have a look and test it. Are you sure you have also uploaded template before pushing data? Another thought is what version of elasticsearch do you use?

from apachebeat.

I am currently testing with docker-elk, which uses the latest version (2.2.0),
I have tried uploading the template before pushing data. But in my current configuration I am using logstash with the beat input, which might be a source of problems

from apachebeat.

If possible, could you test it also without Logstash? I am almost 100% sure it is caused by logstash.
Do you use the same name for the index? If not, then logstash pushed its own template and this will be most likely the cause of the issue.

from apachebeat.

I've now also tested it with direct output to ES and the problem still persists.

from apachebeat.

Just to make sure, you have to perform test with empty elasticsearch, with no index and template. If you only change output but elasticsearch has already data in it, it will not update field types.

from apachebeat.

apachebeat sends strings according to the source

cpu_load     string  
uptime       int  
req_per_sec  string

from apachebeat.

Yes, this is true.
Template which is imported to empty elasticsearch is parsing string as float in the end as defined in template mapping. But once it is already in elasticsearch as string, then it is not parsed as float.
JSON format is also text parsed by Elasticsearch and if correct template is imported, then all work well.

OK, but honestly, apachebeat should sent float in it. I take it.
Meanwhile I am working on a fix, which will convert strings correctly.

from apachebeat.

it looks like you fixed the issue for the beats repository, can you apply the fix here also ?
or are you intending to depricate this package in favor of beats ?

from apachebeat.

true, I do work on porting apachebeat to beats and implement best what I have learned so far :)

Work is in progress and I will apply fix also here, but I have to change some part of the mechanism of the apachebeat. I want to reflect also changes of fieldnames and more to be as close as possible and usable with same templates.

I am not decided yet, if I deprecate apachebeat in future. If there is community drive using it as standalone beat, I will develop it further. Let's see what will happen :)

from apachebeat.

Note, apachebeat is working just fine, if you import mapping first. Then ES handles float values correctly even coming from beat as string.

from apachebeat.

As stated obove, I applied the mappings on a fresh elk-stack before starting apachebeat.
I still get the following errors in Kibana

  Expected numeric type on field [apache.cpu_load], but got [string] 
  Expected numeric type on field [apache.req_per_sec], but got [string]

from apachebeat.

@borian I have committed complete rework (simplification) of apachebeat to fix issues with float types and also changes according to metricbeat to have the (almost) same structure. If you have time to test branch: simplification. I will appreciate any feedback. Work is still in progress in this branch.

from apachebeat.

types are working now! thanks a lot
I still can't view the graphs, it shows no results found, maybe some type missmatch

but I can't look into it right now
but thanks again for your work !

from apachebeat.

@borian you have to use also new visualizatuons since the structure has been changed as in metricbeat.

from apachebeat.

No, I forgot. Thanks for the follow-up
On 29 Mar 2016 17:51, "Radovan Ondas" [email protected] wrote:

@borian https://github.com/borian you have to use also new
visualizatuons since the structure has been changed as in metricbeat.

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#17 (comment)

from apachebeat.