monitoring_tool's People
monitoring_tool's Issues
Min/Max values
Output one min/max value per interval.
Output a curve.
Double print on console
20190723 10:34:06 - 0.001888 0.000722 0.003263 0.600000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:06 - 0.001888 0.000721 0.003263 0.600000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:11 - 0.002062 0.000711 0.003490 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:11 - 0.002062 0.000711 0.003490 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:16 - 0.002133 0.000782 0.003179 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:16 - 0.002133 0.000783 0.003180 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:21 - 0.002613 0.001238 0.003393 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
20190723 10:34:21 - 0.002613 0.001238 0.003393 0.800000 0.000000 1.000000 0.000000 0.000000 0.000000 1.000000 0.000000
Http request type CDF
Example output:
Get request received is x% from all received requests
Function Inputs:(dest IP, dest port, src IP, Src port, monitoring time window)
Refactor hashtables and implement client/path count and popular items
Check the message type if we expect other msg than init!!
server.c - connection_handler
Extract Method or file name from the URL path
for example:
https://abc.xyz.com/path/somefilename.xy?param1=blie¶m2=bla
https://abc.xyz.com/path/somefilename.xy
https://abc.xyz.com/path/somefilename.xy/
https://abc.xyz.com/path/logout
https://abc.xyz.com/path/search/?param1=blie¶m2=bla
https://abc.xyz.com/path/search?param1=blie¶m2=bla
Display the CDF graph of the most popular X requested objects(e.g:somefilename.xy ) or the method (search or logout).
Seg fault with a lot of traffic
Http response status CDF
Example output:
Responses with 200 is X% of the the total returned responses
Responses with 304 is Y% of the the total returned responses
Function Inputs: (dest IP, dest port, src IP, Src port monitoring time window)
add measurement code at run time
Refactor the min/max system
Output one min/max value per interval.
Output a curve.
Some rsp cannot be matched with req
Check if analysis results are sent to the correct client
onExit - check if we can close all clients
server.c
Create a bash script to automate the client/server setup
Implement the throuput
- direction (1 graph with 6 curves), min/max/avg
- move the client filter for all the metrics
- number of bytes (tcp payload size)
Name the number of clients as number of connections and group by different connection Src Ip to get the expected number of clients
Prevent the sniffer to die when the interface doesn't exists
TODO
1 -Global To
2- Standalone function for each metric.
- input/output
- with r w/o params
3 -min/max refactoring
4 -ST
- for all client : destination
TP
a specific From
num of clients
cumulative cdf for request path
parametrable number
flow packet cleanup
Throughput
- specify a client and server
- chose the direction(req/rsp)
- number of bytes/s
- min/max/avg
Usage of GP_CAP_FIN - discuss with Mona
main.c
int GP_CAP_FIN = 0
I needed to substitute this global var for a shared var among thread. Want to discuss if this change can be problematic.
Correct ws pb with client (send data results per client)
modify the sniffer to accept mysql traffic and put in a separate mysql hash-table
Send a close message
The sniffer should send a close message to the front-end once the analysis is done
Make the server persistent
Restart it on failure
Server - add a threadPool?
Some min error rate values are 0
Return the formation of HTTP pairs based on the null requests in the session
remove the validation based on request seq_next and response ack
Implementing rw lock
Test the sniffer performance with pcap_next timeout set to -1
Measureing the cpu and memory utilization difference for the sniffer between the following two scenarios:
pcap_next paramter is set to 1000 ms
pcap_next paramter is set to -1
Review the server implementation
Diagram and breif description of the system architecture
The components (front end, back end and sniffer), the technology of each one, the communication protocol along with the test application (YCSB) components.
if we start/quit/start ycsb, we have constantly increasing min/max rst value
Documentation
System architecture (Components/ communication protocols used/ function(s) of each componet)
Installation guide/configuration/environment of each component along with issues met and how are resolved
code structure and documentation.
pcap_next block
Lock are not efficient. Doesn't allow 2 concurent different writes
data.c
We can maybe remove them, as only one thread write these values.
One lock per variable?
- rst
-rstmin
...
Check memory leak
in main.c - capture_main
//TODO: Check with this is commented! Memory leak.
// Is it to solve the seg Fault?
//packet_free(packet);
Be sure to lock before editing global variable
#9 Loss of packets
Check how to handle the offline mode
Change was made in
- process_packet_queue
- process_flow_queue
#2 - the rst is sometime negative
We notice a problem with the response stamp (greater than the request's one)
Review the lock system for the hash_table counts
Uniformize error handling on the sniffer
modify the sniffer to accept memcache traffic and put in a separate memcache hash-table
Check if malloc is necessary
server.c - extract_data
For the saddr/daddr allocation.
#1 - the rst is only computed when the connection is closed
The flow is moved from the hash_table to the flow_queue once the connection is closed. This is a problem when the connection is persistent.
The timeline is not the same for all metrics
- add a division for the interval
- add the axis unit
#5 - rst avg is below rst min
double request count
Ensure different measurement functions implemented as separate modules
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.