Comments (7)
@ticoann confirmed that the default value event time is -1 if there is no value provided. It is obvious that WMArchive integration counted the default value -1. The purpose of -1 instead of 0 is because this give us a way to know if there is an event time with value 0 or none. The correct integration is not counting the -1's. @vkuznet will look into the integration code to see if there is a way to systemically ignore the default.
from wmarchive.
Hi,
I made a change to count only positive values. Could you please look-up data again on testbed and see if things change. I verified that one of the Dima's negative plot now show almost zero avg. time, see http://bit.ly/2fWu1hG
from wmarchive.
The code changes are here: #252
from wmarchive.
I verified that one of the Dima's negative plot now show almost zero avg. time
This sounds too small. But its seems there is no 0 values. Are we using directly what is reported or changing the unit?
from wmarchive.
Seangchan,
there is no data manipulation in wmarchive, we see is what what we store.
I only applied a cut on values to be greater-equal than zero.
Valentin.
On 0, ticoann [email protected] wrote:
I verified that one of the Dima's negative plot now show almost zero avg. time
This sounds too small. But its seems there is no 0 values. Are we using directly what is reported or changing the unit?
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
#245 (comment)
from wmarchive.
@vkuznet
@ticoann
Below plot still have a 1-2 second/event and it seems to small according to Diam, but I think it is correct. Do you?
If I choose log collect as job type, there is no event time. If I choose merge type , this is almost zero event time. The processing type has a regular 10s/event. So if I average them together, log collect will not contribute, but average processing and merge will make it much small.
Eventthrough would be more useful than event time. But we don't have it. why?
from wmarchive.
Seangchan @ticoann please confirm that this is solved.
from wmarchive.
Related Issues (20)
- Investigate spark streaming approach HOT 1
- Implement daily dir structure for avro files HOT 1
- wmaClient HOT 1
- don't allow duplicate docs get inserted. HOT 1
- FWJR performance documentation HOT 6
- Add filter for aggregation getting only final fwjr HOT 4
- Document keys used in WMArchive HOT 3
- UI request from Jen HOT 13
- Archive data unit change HOT 25
- Integration tests to understand the ExitCodes HOT 37
- Paola's usecase HOT 11
- Actions needed to have the next release as Nov 30 HOT 5
- Adjust file size to be less of 256MB for HDFS migration HOT 2
- Reduce latency in aggregation HOT 5
- Need to pull logs multiple ways and compare them HOT 16
- Update avro input path
- Missing required field PrepID HOT 18
- Large queries with WMArchive HOT 6
- Investigate and fix source of errors in WMArchive HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from wmarchive.