Giter Club home page Giter Club logo

node-sflow's Introduction

node-sflow

SFlow compatible library

The library is still under development, please be careful! It has been tested with Extreme XOS only! Please raise issues in case of a problem!

Usage

The usage of the Sflow collector library is very very simple. You just have to do something like this:

var Collector = require('node-sflow');

Collector(function(flow) {
    console.log(flow);
}).listen(3000);

Keep in mind that even SFlow is a very powerful protocol (in many cases better than NetFlow, even if it is compared to NetFlow version 9) the implementation of the protocol is usually very limited by the hardware vendor.

This module only decodes to JSON the SFlow information provided by the SFlow source. If that is a decent source, the packet information, including L2, L3, L4 properties will be present as SFlow properties and you will have it decoded to JSON by this module.

However, many simple Ethernet switchies does not really implement SFlow. They just use it as a transport protocol and just encapsulate (usually the first 64 bytes of an) Ethernet packet on top of SFlow container and then send it to the collector. If your case is this one, then you will just receive Ethernet packet as raw data and you will not have the L2, L3, L4 properties decoded and you have to decode the Ethernet packet on your own.

Luckily, decoding raw Ethernet (or other) packets into JSON is realtively easy task, as there are a lot of NPM modules you could use to do that. Node.JS NPM provides you with a lot of helpful tools for it. I am sure you will find the best one fitting your needs.

If you are confused, you can look at this simple example, where I use the pcap module to decode raw ethernet packets received over SFlow (the following example uses the pcap module for Node.JS 0.12 to decode the ethernet fames):

var Collector = require('node-sflow');
var pcap = require('pcap');
Collector(function(flow) {
    if (flow && flow.flow.records && flow.flow.records.length>0) {
        flow.flow.records.forEach(function(n) {
            if (n.type == 'raw') {
                if (n.protocolText == 'ethernet') {
                    try {
                        var pkt = pcap.decode.ethernet(n.header, 0);
                        if (pkt.ethertype!=2048) return;
                        console.log('VLAN',pkt.vlan?pkt.vlan.id:'none','Packet',pkt.ip.protocol_name,pkt.ip.saddr,':',pkt.ip.tcp?pkt.ip.tcp.sport:pkt.ip.udp.sport,'->',pkt.ip.daddr,':',pkt.ip.tcp?pkt.ip.tcp.dport:pkt.ip.udp.dport)
                    } catch(e) { console.log(e); }
                }
            }
        });
    }
}).listen(3000);

In the above example, I use an integrated feature in the node-pcap module to decode the raw packet content. However, node-pcap module currently works only with Node.JS 0.10-0.12 and do not support the new C++ interface introduced in Node.JS 4 and 5. If you want to use Node.JS 4 and 5, try node-pcap2 module there. The API is a bit different (the decoder expects PCAP header too), but is not as much different. The following example works with node-pcap2 module for Node 4 and 5:

var Collector = require('node-sflow');
var pcap = require('pcap2');
Collector(function(flow) {
    if (flow && flow.flow.records && flow.flow.records.length>0) {
        flow.flow.records.forEach(function(n) {
            if (n.type == 'raw') {
                if (n.protocolText == 'ethernet') {
                    var pcapDummyHeader = new Buffer(16);
                    pcapDummyHeader.writeUInt32LE((new Date()).getTime()/1000,0); // Dummy time, you can take it from the sflow if you like
                    pcapDummyHeader.writeUInt32LE((new Date()).getTime()%1000,4);
                    pcapDummyHeader.writeUInt32LE(n.header.length,8);
                    pcapDummyHeader.writeUInt32LE(n.frameLen,12);
                    var pkt = pcap.decode.packet({
                       buf: n.header,
                       header: pcapDummyHeader,
                       link_type: 'LINKTYPE_ETHERNET'
                    });
                    if (pkt.payload.ethertype!=2048) return; // Check if it is IPV4 packet
                    console.log('Packet',pkt)
                }
            }
        });
    }
}).listen(3000);

node-sflow's People

Contributors

delian avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

node-sflow's Issues

Question re license

Hi there,

Would you consider the LGPL or BSD license for your project? It's a great piece of work and seems to work very nicely, I've gone ahead and made a proposed (work in progress) prototype for SFlow support in the node-logstash project (https://github.com/bpaquet/node-logstash) bpaquet/node-logstash#102. This will allow logging SFlow records straight into an ELK stack.

Hope you will consider this request.

Kind regards,

Q: If a packet have several flows, isn't it always overwritten with the last one?

In sflow.js there are a loop over the samples and that stores the flow in the parameter o.flow.
That parameter is owerwritten in each loop, wouldn't that mean that you would only get the last one?
If you pass on a callback you would of course get each with the callback but if you don't...

for (var n = hdr.samples;n;n--) {
    var sHdr = buf.readUInt32BE(0);
    o.flow = {};
    ....

unknown format error

Hi I'm using host-sflow will the following config:
pcap { dev = docker_gwbridge }

My js code has this:
Collector(function(flow) {
if (flow && flow.flow.records && flow.flow.records.length>0) {
flow.flow.records.forEach(function(n) {
console.log(n);
});
}
}).listen(6343);

After a few flows, it consistently fails with "unknown format" err. Following is out with debug turned on:

[root@ip-10-227-81-88 scripts]# DEBUG=sflow,sflow:packet node swarm-metrics.js
sflow got a packet +0ms
sflow:packet header: {"sflowVersion":5,"ipVersion":1,"ipVersionText":"IPv4","ipAddress":"10.227.81.88","subAgentId":100000,"sequence":14,"uptimeMS":81255,"samples":5} +3ms
sflow:packet sample 0:{"enterprise":0,"format":1,"length":112} +1ms
sflow:packet reading flow 0 with enterprise 0, format 1 and length 72 +0ms
sflow gow a flow 25 from packet with sequence 14 +1ms
sflow:packet sample 1:{"enterprise":0,"format":1,"length":112} +7ms
sflow:packet reading flow 0 with enterprise 0, format 1 and length 72 +0ms
sflow gow a flow 26 from packet with sequence 14 +0ms
sflow:packet sample 2:{"enterprise":0,"format":1,"length":184} +2ms
sflow:packet reading flow 0 with enterprise 0, format 1 and length 144 +0ms
sflow gow a flow 27 from packet with sequence 14 +0ms
sflow:packet sample 3:{"enterprise":0,"format":1,"length":112} +0ms
sflow:packet reading flow 0 with enterprise 0, format 1 and length 72 +0ms
sflow gow a flow 28 from packet with sequence 14 +0ms
sflow:packet sample 4:{"enterprise":0,"format":1,"length":184} +1ms
sflow:packet reading flow 0 with enterprise 0, format 1 and length 144 +0ms
sflow gow a flow 29 from packet with sequence 14 +0ms
sflow got a packet +7s
sflow:packet header: {"sflowVersion":5,"ipVersion":1,"ipVersionText":"IPv4","ipAddress":"10.227.81.88","subAgentId":100000,"sequence":15,"uptimeMS":81255,"samples":1} +0ms
sflow:packet sample 0:{"enterprise":0,"format":2,"length":680} +0ms
sflow:packet unknown format +1ms { format: 2001, length: 4 }
/etc/chef/compose-stacks/sflow-rt/node_modules/node-sflow/lib/packet.js:335
throw new Error('unknown format');
^

Error: unknown format
at Object.internal.readCounterRecords (/etc/chef/compose-stacks/sflow-rt/node_modules/node-sflow/lib/packet.js:335:23)
at /etc/chef/compose-stacks/sflow-rt/node_modules/node-sflow/lib/packet.js:428:50
at _combinedTickCallback (internal/process/next_tick.js:131:7)
at process._tickCallback (internal/process/next_tick.js:180:9)
[root@ip-10-227-81-88 scripts]#

Any idea what is happening here?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.