Giter Club home page Giter Club logo

nfsen-ng's Introduction

Hi there, I’m Michael πŸ‘‹

  • Self-employed Full Stack Software Engineer – I work with PHP, TypeScript and everything in-between πŸ’»
  • Back from a 16 month roadtrip with our '92 Mercedes T1 310 4x4 Van πŸš’
  • Originally hailing from Switzerland πŸ‡¨πŸ‡­πŸ”
  • BSc in Computer Science (2017) with IT-Security as major πŸŽ“
  • Married πŸ’
  • Love doing and discussing
    • cycling 🚴 and running πŸƒ (follow me on Strava)
    • photographing πŸ“· check out some photos or my insta
    • cars πŸš— especially if they’re light and fast
    • music 🎢 have a look at my last.fm
  • My PGP public key is on keybase.io πŸ”‘

Technologies I use almost daily

I’m mainly working on PHP 8 projects and web pages using a self-made framework/boilerplate which is unfortunately private.

  • PHP (using Slim framework v4, Twig, JWTs, Dependency Injection, strictly typed classes :)
  • MySQL/MariaDB
  • ES6 JavaScript and TypeScript (Lit, ESbuild, etc.), sometimes React or Vue.js
  • CSS/Sass (Bulma framework)

Stuff I’d like to use/do/learn more

  • Python
  • Arduino
  • DevOps
  • IT-Security

πŸ‘· Check out what I'm currently working on

  • mbolli/nfsen-ng - Responsive NetFlow visualizer built on top of nfdump tools. (4 months ago)
  • enygma/yubikey - PHP library to interface with the Yubikey REST API (8 months ago)
  • twigphp/Twig - Twig, the flexible, fast, and secure template language for PHP (9 months ago)

🌱 My latest projects

πŸ”­ Latest releases I've contributed to

πŸ”¨ My recent Pull Requests

πŸ““ Gists I wrote

⭐ Recent Stars

❀️ These awesome people sponsor me (thank you!)

Language stats

nfsen-ng's People

Contributors

bobryan530 avatar dehnli avatar dominiquefournier avatar dona21 avatar fontouraabreu avatar jp-asdf avatar luizgb avatar mbolli avatar nrensen avatar panaceya avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

nfsen-ng's Issues

sp and dp not displayed

How to reproduce

  • Nfdump returns a full CSV with the source and destination port of the flows
  • I select any format that contains %dp
    Ports are not displayed in the GUI

Get CSV: returns the CSV with those fields

illegal rrd update

I successfully imported some data with -pf -p pf into rrd.
Now, wen running the daemon, it will perodically log "illegal attempt to update using time 1549010700 when last update time is 1549010700 (minimum one second step)" (the timestamp changes with the latest nfcapd file).

Running on Debian-Stretch with rrdtool 1.6.0 and php-rrd 2.0.1

alerts support

Alerts allow you to execute specific actions based on specific conditions. An alert is defined by a filter applied to the 'live' profile, conditions, triggers and alert actions. The original nfsen does support this.

getting a graph based on a filter

Dear All,

In the "Flows" tab we are able to define some advanced filters. Is there a way to see these results in the same way as graph over time as it is possible in the "Graphs" tab ?
Or to take over the filter definition into the graphs result ?
I couldn't find anything.

Output type "Custom" could be a solution. But no idea what the "custom output format" should be.

Kind regards
Hans

--

Selecting a single 5-minute bucket - 503 error

(Expansion of issue mentioned in #31)

One of the things I do most often in nfsen is to move the marker to a single 5-minute bucket. For example, see the peak at 20:55 (UTC) here:

image

It's very easy then to run queries on this bucket in the bottom half.

Using nfsen-ng, the same peak exists (note: this time it's showing local time, BST, so it's at 21:55)

image

So I then drag the start and end boundaries as close as I can around the peak:

image

and switch to "Statistics" - but unfortunately I get a 503 error when running the query.

image

It does work if I select a larger time range, say 15 minutes, but that dilutes the query somewhat.

So I see two issues here:

  • work out why the 503 error is occuring, and prevent it (it's reproducible)
  • provide in the UI a convenient way to select a single 5-minute bucket. For example, hovering over the graph shows a big dot on the point, but clicking on it currently doesn't do anything - so a click to select the time range that the point covers would be great.

why not

Got 503 - Service unavailable. NfDump: Internal error. stat() error '/var/nfdump/profiles_data/live/source1/2019/07/31/nfcapd.201907310405': Permission denied

Expire and Max Size

Hello,
Can i configure analog of parameter 'Expire' and 'Max size', its mean how long we can store data before auto deleting?

Aggregation in "Statistics" doesn't seem to do anything

This is with nfsen-ng master (27b7365) under Ubuntu 18.04.

In the Statistics view, the aggregation buttons don't seem to do anything - they don't change the command sent to nfdump. To reproduce:

  • Go to Graphs tab, select last 24 hours
  • Go to Statistics tab, click "Process data"

The nfdump command it generates is:

 nfdump command: /usr/local/bin/nfdump -M '/var/nfsen/profiles-data/live/gw' -R '2020/06/11/nfcapd.202006110825:2020/06/12/nfcapd.202006120810' -n '10' -o 'csv' -s 'record/flows' 2>&1

Then select "Global Aggregation - Bidirectional", and click "Process data" again, it gives exactly the same command:

nfdump command: /usr/local/bin/nfdump -M '/var/nfsen/profiles-data/live/gw' -R '2020/06/11/nfcapd.202006110825:2020/06/12/nfcapd.202006120810' -n '10' -o 'csv' -s 'record/flows' 2>&1

Unselect Bidirectional, and select IP Aggregation > Source > IP then "Process data". Exactly the same result.

If I enter a filter, like "net 10.0.0.0/8", that is passed to the backend - but the other buttons still don't do anything.

I have checked the traffic with tcpdump. With the "bidirectional" button selected I see

GET /nfsen-ng/api/stats?datestart=1591863845&dateend=1591950365&sources%5B%5D=gw&filter=&top=10&for=record%2Fflows&title=Flow%20Records&aggregate=bidirectional&limit=&output%5Bformat%5D=line&output%5Bcustom%5D= HTTP/1.1

With IP Aggregation > Source > IP selected I see

GET /nfsen-ng/api/stats?datestart=1591863845&dateend=1591950425&sources%5B%5D=gw&filter=&top=10&for=record%2Fflows&title=Flow%20Records&aggregate=srcip&limit=&output%5Bformat%5D=line&output%5Bcustom%5D= HTTP/1.1

Clearly the correct flags are making it into the HTTP request, but for some reason the backend is ignoring them.

Aside: if you select "Statistic for" anything other than "Flow Records", then those buttons are greyed out anyway (and therefore not expected to do anything)

how to add multiple NetFlow /sflow sources

My NetFlow station build around Nfsen works well , however I am not sure how to add multiple Sflow/netflow sources on same port in Nfsen-ng.

please suggest where to add source IPs and ports into the configuration, in Nfsen-ng there is no option to add IPs in backend/settings.php for multiple sources. please help, I am very impressed with the web front-end of Nfsen-ng and would like to start a new deployment with Nfsen-ng.

Add more tooltips

For new users it might help to explain certain fields or functions using the tooltip functionality.
screenshot_2017-05-28_13-30-10

file settings.php cannot be read

The interface does not display any parameters that are not built in the file

ls -l settings/
total 8
-rw-r--r-- 1 http http 1004 Feb 27 10:22 setting.php.orig
-rw-r--r-- 1 http http  995 Feb 27 12:08 settings.php

What have I done wrong?

load/store nfdump filters

In the flows or statistics view, make it possible to save a filter (either on the server, or using localStorage or similar) and load previously saved filters.
screenshot_2017-05-28_13-26-08

Report on all ports

Hi,

Is it possible to review traffic on all ports, without having to specify each one?

Every evening/night my internet usage spikes for a couple of hours, burning up to 20 GB of data (I would generally see 2 GB for the rest of the day). I have been had nfcapd running for the past several days.

At the moment my nfsen-ng configuration file has ports 80 and 443 specified, but can't see the usage spikes in the data, so assume it may be on another port, but don't know which ports to even try.

Any thoughts would be appreciated.

503 error on subnet aggregation with missing prefix length

(Expansion of issue from #31) This is a minor usability / data validation issue. To reproduce:

  • Go to Graphs tab, select last 24 hours
  • Go to Flows tab
  • Under IP Aggregation > Source select "IPv4 subnets", but leave the prefix length empty
  • Click Process Data

Result is a red 503 error:

image

Got 503 - Service unavailable. NfDump: Initialization failed. /usr/local/bin/nfdump -M '/var/nfsen/profiles-data/live/gw' -R '2020/06/11/nfcapd.202006110810:2020/06/12/nfcapd.202006120805' -c '20' -o 'csv' -a '-Asrcip4' 2>&1

The problem is fixed if you type (say) "24" in the prefix length box.

I think the issues are:

  • if nfsen-ng shows grey "/24" in the box, the user assumes that /24 is already taken as default
  • if you type "/24" it fails since this is only a 2-digit box; then you realise you have to enter "24"

I think it would be better if the slash were moved out of the selection boxes:

      [ IPv4 Subnets     ] / [   ]

The box can display a grey "24" (making it clearer what the user is expected to enter here). Then if the user leaves this blank, 24 can be supplied to the backend.

Function ReflectionType::__toString() is deprecated

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 64

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 64

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 64

Deprecated: Function ReflectionType::__toString() is deprecated in /srv/http/nfsen-ng/backend/api/api.php on line 60

PHP 7.4.3

bit of a newbie question(s)

Hi, I'm relatively new to nfsen and have gotten to a point where I stuck. Sorry if this is very basic ... Through lots of reading I have nfcapd running creating files under /var/nfdump/profiles-data/live/bachman - i started it with

nfcapd -w -D -p 2025 -u netflow -g www-data -B 200000 -s 1 -z -I bachman -l /var/nfdump/profiles-data/live/bachman

Now onto where I'm stuck ... nfcapd has been running for about 10 hours so I would expect to see lots of data ... but this is what I get when I go to the browser ....

Screen Shot 2020-03-27 at 11 39 24 AM

And when I try ./cli.php status I get no response, it also says that a log should be created in /var/www/html/nfsen/backend ... but it's not - so what am i missing ??

Also in your readme, while it describes how to install nfsed-ng maybe a couple of lines on adding the netflow user and how to start nfcapd would be useful

thanks in advance
Ken

nfsen-ng: 400 - Bad Request. rrd_xport failed.

Dear All,

I am not sure if I am right here. I didn't find a mailing for "nfsen-ng"

My issue I don't get any data on the GUI.

I followed the instructions at https://github.com/mbolli/nfsen-ng to install on a fresh installed "Ubuntu 18.04.1 LTS"
"nfdump" wasn't found as packaged, so I compiled from source.

This is my file nfsen-ng/backend/settings/settings.php

<?php
/**
 * config file for nfsen-ng
 *
 * remarks:
 * * database name = datasource class name (case-sensitive)
 * * log priority should be one of the predefined core constants prefixed with LOG_
 */

$nfsen_config = array(
    'general' => array(
        'ports' => array(
            80, 22, 53,
        ),
        'sources' => array(
            'cs-sl0-s6506',
        ),
        'db' => 'RRD',
    ),
    'frontend' => array(
        'reload_interval' => 60,
    ),
    'nfdump' => array(
        'binary' => '/usr/bin/nfdump',
        'profiles-data' => '/var/nfdump/profiles-data',
        'profile' => 'live',
        'max-processes' => 2, // maximum number of concurrently running nfdump processes
    ),
    'db' => array(
        'Akumuli' => array(
            //'host' => 'localhost',
            //'port' => 8282,
        ),
        'RRD' => array()
    ),
    'log' => array(
        'priority' => LOG_DEBUG, // LOG_INFO, LOG_DEBUG is very talkative!
    )
);


I run

nfcapd -w -D -S 2 -B 1024000 -l /var/nfdump/profiles-data/live/cs-sl0-s6506 -p 10001

which writes sucessfully data in subdiretories like this

/var/nfdump/profiles-data/live/cs-sl0-s6506/2018/11/13/16/nfcapd.201811131620

But when I open my web-browser I get an error message:

Got 400 - Bad Request. rrd_xport failed. opening '/var/www/html/nfsen-ng/backend/datasources/data/cs-sl0-s6506.rrd': No such file or directory

and syslog shows at the same time:

Nov 13 16:52:52 hpg911 apache2: nfsen-ng: Was not able to find /var/www/html/nfsen-ng/backend/datasources/data/cs-sl0-s6506.rrd
Nov 13 16:52:52 hpg911 apache2: nfsen-ng: 400 - Bad Request. rrd_xport failed. opening '/var/www/html/nfsen-ng/backend/datasources/data/cs-sl0-s6506.rrd': No such file or directory

I do not know where to configure a process to convert the pcap data to rrd files.
I gave full permissions for /var/www/html/nfsen-ng/backend/datasources/data/
so it shouldn't be a permissions issue.

RRDtool 1.7.0 is installed,

apachectl -M shows php7_module is installed, but nothing about rrd, probably OK.

"pecl list" tells me

rrd 2.0.1 stable

I started manually "./cli.php start"

./cli.php status
Running: 14178

Any help is welcome.

// Hans

Error parsing "profile-data" directory

Hello

I am unable to import historic data. The following error message is received.

Processing 1 sour...33.2% 364/1097 ETC: < 1 sec. Elapsed: < 1 sec [=========> ]
0.2758s /var/nfdump/profiles-data//*/2016/08/28 does not exist!

The directory structure does however contain the files and directory structure
/var/nfdump/profiles-data/2016/08/01 -> /var/nfdump/profiles-data/2016/08/30

Please assist

timezone problem

i want to know how to set timezone, the graph time is later than local time...my timezone is utc+08:00

image

NFDUMP filter "-o nel" does not work

Hello, i try to use the nfdump filter "-o nel" to see Nat Events with nfsen-ng. If i try to use the NFDUMP-Filter text field under Flows the filter is added wrong to the nfdump command.

Output:
/usr/local/src/nfdump-1.6.20/bin/nfdump -M '/var/flows/rt1.mn.coe' -R '2020/06/17/nfcapd.202006170845:2020/06/18/nfcapd.202006182240' -c '20' -o 'csv' -a '-Asrcip' '-o nel'

Correct would be:
/usr/local/src/nfdump-1.6.20/bin/nfdump -M '/var/flows/rt1.mn.coe' -R '2020/06/17/nfcapd.202006170845:2020/06/18/nfcapd.202006182240' -c '20' -o 'csv' -a '-Asrcip' -o 'nel'

I am not sure if this is really an issue or an enhancement request.

thanks,
glueck

settings.php

Didn't find proper documentation for settngs.php.
I'd like to fetch data from my nfcapd files. Which type of DB i should use?
What ports section stands for?

$nfsen_config = array(
'general' => array(
'ports' => array(
80, 22, 53,
),
'sources' => array(
's1', 's2',
),
'db' => 'RRD',
),

Limit time window for flows and statistics

When some kind of aggregation comes into play, nfdump will rather soon get very slow. Depending on the system, even getting the top flows for one week might take more than half an hour (and using >15 GB RAM in the process).
It might make sense to forcefully limit the maximum delta to something around one week, or user-configurable in the settings file.

Kill running nfdump process from front end

As nfdump processes might take a long time to complete (depending on the parameters and the system), it would make sense to somehow track the running processes and make them killable from the front end.

Drop "full ipv6" flag

nfsen-ng uses -o csv to get structured data output of nfdump. ipv6 addresses are always shown in full format, which makes this flag obsolete.

Originally posted by @mbolli in #53 (comment)

add time option to flows

Hi,

brilliant code and wonderful panel!

im wondering how simple it would be to add a time option to the flow tab?
currently selecting 24 hours is brilliant to get todays event,
and the slider is brilliant to select say another day like 2 days ago,

but i then have to scroll threw pages and pages and pages just to find the rough time i needed to check traffic

all you would have to do would be use this extra command for nfdump
-t 2020/05/26.18:15:00-2020/05/26.18:30:00
this would then search for data thats only between those timestamps!

nfcap/dump not starting

Installed on Ubuntu 18.04, nfsen-ng is running, no errors in the browser anymore, but I don't see any nfcapd processing running, or files created in my /var/nfdump/profiles-data/live directory. If I manually start nfcapd it works, and I can see flow data.

Not sure what i'm missing, no errors via browser or log file pointing me to a problem.

Thanks for the help.

can't start ./cli.php

i'm trying to start the deamon but i've this error πŸ‘
/var/www/html/nfsen-ng/backend/common/config.php:20 Stack trace: #0 /var/www/html/nfsen-ng/backend/cli.php(6): common\Config::initialize() #1 {main} thrown in /var/www/html/nfsen-ng/backend/common/config.php on line 20
I've set my settings.php like this :
$nfsen_config = array( 'general' => array( 'ports' => array( 80, 22, 53, ), 'sources' => array( 'test1', ), 'db' => 'RRD', ), 'nfdump' => array( 'binary' => '/usr/local/bin/nfdump', 'profiles-data' => '/home/netflow/test', 'profile' => 'live', 'max-processes' => 1, // maximum number of concurrently running nfdump processes ), 'db' => array( 'Akumuli' => array( 'host' => 'localhost', 'port' => 8282, ), 'RRD' => array() ), 'log' => array( 'priority' => LOG_INFO, // LOG_DEBUG is very talkative! ) );

I'm surely doing something wrong but i can't find out,
thanks in advance for help

Error creating rrd file: not writable

Hello,
I am trying to get nfsen-ng to work on Ubuntu 18.04 system. I followed the instructions but when I am trying to import nfdump files I get the message (in /var/log/syslog)
nfsen-ng: Error creating /var/www/html/nfsen-ng/backend/datasources/data/1234.rrd: Not writable
All the directories from /var/www down are owned by www-data and have rw privileges for www-data.
What am I doing wrong?

No data in Web UI, but rrd files are generated

Hi
I manage to get nf dump files generated, and also the rrd files are being generated and updated.
Currently I have both the basement_[port].rdd files and the [port].rdd files being updated.
I am running the newest version from 17 of July this year

Is there a way to check if the rrd files contain any data?

What could be the reason why I am not seeing anything on the web UI.
When I try the default values for 24H under the statistics it show the following on the web ui:
image

Settings file

<?php
/**
 * config file for nfsen-ng
 *
 * remarks:
 * * database name = datasource class name (case-sensitive)
 * * log priority should be one of the predefined core constants prefixed with LOG_
 */

$nfsen_config = array(
    'general' => array(
        'ports' => array(
           80,443 ,
        ),
        'sources' => array(
            'basement',
        ),
        'db' => 'RRD',
        'processor' => 'NfDump',
    ),
    'frontend' => array(
        'reload_interval' => 60,
    ),
    'nfdump' => array(
        'binary' => '/usr/local/bin/nfdump',
        'profiles-data' => '/var/nfdump/profiles-data',
        'profile' => 'live',
        'max-processes' => 1, // maximum number of concurrently running nfdump processes
    ),
    'db' => array(
      //  'Akumuli' => array(
            //'host' => 'localhost',
            //'port' => 8282,
       // ),
        'RRD' => array()
    ),
    'log' => array(
        'priority' => LOG_DEBUG, // LOG_DEBUG is very talkative!
    )
);

Preferred views

To make life simpler in normal operation, I would prefer:

  • the default graph view to be "Traffic" rather than "Flows"
  • the default statistics "order by" to be "Bytes" rather than "Flows"
  • traffic to be shown as bits/sec rather than bytes/sec

I realise that not everyone shares my preferences, but perhaps these could become options in settings.php?

nfsen features / problems for an nfsen user

Hello,

I have nfsen-ng installed. Here are some things I'm having problems with but was able to do in nfsen. It could be simply due to operator error and I can't work out how to do it.

Single time bin queries

In nfsen, it was easy to move the slider to point to any single 5-minute "bin" - e.g. where there's a nice peak - and then do a query below it.

image

I can't see how to do this in nfsen-ng. There is a very coarse slider at the top (3 year period), and given the graph below there's a small zoom slider below that, which I can zoom on in stages. But changing the lower zoom slider doesn't carry forward when I switch from "Graphs" to "Flows".

On the "Graphs" page there's a button "Copy from graph", but it's greyed out. When I switch to "Flows" this button vanishes completely.

Range selection

Related to the above: in the "Flows" page there's only the coarse slider at the top. I can switch to "24 hours" and move back and forth one day at a time using the arrows; but I can't see how to say, for example, "I'm only interested in 21:10 to 22:30"

In nfsen, I could just split the time indicator, and move the start and end positions to 21:10 and 22:30, and queries would cover that range.

Time period display

This is more just an observation than a problem.

In the Graph page, it took me a while to work out how to zoom in far enough: I had to first click the "24 hour" button at the top.

Here is the peak as seen by nfsen-ng, corresponding to the peak at 18:35 of 34.2 Mbps in the nfsen screenshot earlier:

image

It shows as 4.2MB/sec (which is the same). However I don't really like the "ramp up, ramp down / join-the-dots" line; I think that nfsen is more accurate by showing it as a bar, covering the time period ("the average value from 18:30 to 18:35 was X") rather than a point ("the value at 18:35 was X").

Having said that, nfsen is not always precise with start/end times either. It displays "t(start) 18:35, t(end) 18:35" when I think it means "t(start) 18:30, t(end) 18:35".

Translated source/destination

In nfsen I can view NAT-outside (translated) source and destination addresses; I can't see them in nfsen-ng.

** nfdump -M /var/nfsen/profiles-data/live/gw  -T  -r 2019/11/14/nfcapd.201911141835 -c 20
nfdump filter:
any
Date first seen          Event  XEvent Proto      Src IP Addr:Port          Dst IP Addr:Port     X-Src IP Addr:Port        X-Dst IP Addr:Port   In Byte Out Byte
2019-11-14 18:34:44.130 INVALID  Ignore TCP     128.223.157.25:443   ->   XX.XXX.XXX.XXX:47408   128.223.157.25:443   ->      10.12.253.2:47408    13629        0
2019-11-14 18:34:44.440 INVALID  Ignore TCP      10.12.254.107:40183 ->    52.25.210.146:443     81.174.145.162:40183 ->    52.25.210.146:443        181        0
2019-11-14 18:34:44.690 INVALID  Ignore TCP       13.107.18.11:443   ->   XX.XXX.XXX.XXX:60210     13.107.18.11:443   ->    10.12.254.123:60210       88        0
2019-11-14 18:34:44.690 INVALID  Ignore TCP      10.12.254.123:60210 ->     13.107.18.11:443     XX.XXX.XXX.XXX:60210 ->     13.107.18.11:443         40        0

Aside: nfsen doesn't let you aggregate on those fields, unless you patch the PHP slightly:

root@nfsen:/var/www/html/nfsen# diff -u details.php.orig details.php
--- details.php.orig	2018-06-24 16:07:11.246513005 +0000
+++ details.php	2018-06-25 09:01:51.294234237 +0000
@@ -27,7 +27,9 @@

 $IPStatOption = array ( 'Flow Records',
 						'Any IP Address', 'SRC IP Address', 'DST IP Address',
+						'Xlat SRC IP Address', 'Xlat DST IP Address',
 						'Any Port', 'SRC Port', 'DST Port',
+						'Xlat SRC Port', 'Xlat DST Port',
 						'Any interface', 'IN interface', 'OUT interface',
 						'Any AS',  'SRC AS',   'DST AS',
 						'Next Hop IP', 'Next Hop BGP IP', 'Router IP',
@@ -42,7 +44,9 @@

 $IPStatArg	  = array ( '-s record',
 						'-s ip',   '-s srcip',   '-s dstip',
+						'-s xsrcip','-s xdstip',
 						'-s port', '-s srcport', '-s dstport',
+						'-s xsrcport', '-s xdstport',
 						'-s if',   '-s inif',    '-s outif',
 						'-s as',   '-s srcas',   '-s dstas',
 						'-s nhip', '-s nhbip', 	 '-s router',

Minor issues

  1. I was able to get error "Got 400 - Bad Request. Expected type int for datestart" when playing the the zoom slider at the bottom of the Graphs page, especially zooming in towards the latest data (i.e. current time).

  2. If I go to the Flows page and select IP Aggregation - Destination - IPv4 subnets, then it looks like this:

image

The display suggests if I do nothing more, aggregation will be at /24 boundaries. But if I leave the /24 at default, I get an error in the nfdump command:

Got 503 - Service unavailable. NfDump: Initialization failed. /usr/local/bin/nfdump -M '/var/nfsen/profiles-data/live/gw' -R '2019/10/24/nfcapd.201910241950:2019/10/25/nfcapd.201910251940' -c '20' -o 'csv' -a '-Adstip4' 2>&1

Running this command manually shows the actual error:

'dstip4' needs subnet bits too aggregate

Also I can't enter "/24" in this field, it has to be "24" only. Once that's done, it works.

  1. The format 1'649.16 for duration is unusual. Is this 1649.16 seconds? I guess so. (To me it looks like 1 minute 649.16 seconds, but that wouldn't make sense).

data source additional configuration

Hello,

I'm trying to test nfsen-ng in our environment. Can you help me how too specify:

  • netflow sampling rate, eg. from old nfsen: 'switch1' => { 'port' => '53962', 'col' => '#37541d', 'type' => 'netflow', 'optarg' => '-s 2048'}
  • type sflow, eg. from old nfsen: 'switch2' => { 'port' => '52085', 'col' => '#60400d', 'type' => 'sflow'}

Regards,
Michal

Initialization failed with Option -I

Option -I doesn’t have arguments:

   -I Print flow statistics from file specified by -r, or timeslot
      specified by -R/-M.

Error:

Jul  2 14:09:39 nfsen php[22888]: nfsen-ng: Exception: NfDump: Initialization failed. /usr/local/bin/nfdump -I '' -o 'csv' -M '/usr/local/var/nfsen/profiles-data/live/TEST' -r '2019/04/23/nfcapd.201904230345'  2>&1

Incorrect command

 /usr/local/bin/nfdump -I '' -o 'csv' -M '/usr/local/var/nfsen/profiles-data/live/TEST' -r '2019/04/23/nfcapd.201904230345' 

Correct command is:

 /usr/local/bin/nfdump -I -o 'csv' -M '/usr/local/var/nfsen/profiles-data/live/TEST' -r '2019/04/23/nfcapd.201904230345' 

Tested on:

% freebsd-version
12.0-RELEASE-p6
% nfdump -V
nfdump: Version: 1.6.17

nfsen-ng cloned from git e61013a.

Solved: rrd_xport failed. Cannot parse vname

Stumbled upon this after a fresh install on RHEL7. Graph wasn't shown at all in the frontend. API Call to /api/graph response: rrd_xport failed. Cannot parse vname.

After some debugging, I found out this was not an nfsen-ng issue, but rrdgraph related:

Variable names (vname) must be made up strings of the following characters A-Z, a-z, 0-9, -,_and a maximum length of 255 characters.

I was using a FQDN as data source name. This is not supported by RRD libs, and results in error described above.

Mentioning it here to avoid somebody else losing his time with this kind of stuff :) Maybe a warning can be added to the configuration section of the README?

Thanks @mbolli for your great work!

./cli.php status Not running

Hello, i install soft by readme but after configuration i try to start nfsen, but i got error (see bellow)

ipcore@nfseng-ng:/var/www/html/nfsen-ng/backend$ ./cli.php start -v
2020-04-16 06:49:57 CLI: Starting daemon...
int(0)
Daemon running, pid=30296
ipcore@nfseng-ng:/var/www/html/nfsen-ng/backend$ ./cli.php status
Not running

front end: wrong behavior of protocol checkboxes

In the graphs view, when selecting the "protocols" display, it is possible to deselect each protocol. When no protocol is selected, the graph correctly shows the values for "any" protocol. The form should reflect this, i.e. selecting the "any" button.
screenshot_2017-05-28_13-49-14

PHP Notice: Undefined offset

Hi, I keep getting these when trying to import ports:

PHP Notice: Undefined offset: 2 in /var/www/html/nfsen-ng/backend/common/import.php on line 283
PHP Notice: Undefined offset: 1 in /var/www/html/nfsen-ng/backend/common/import.php on line 284

Also, under flows and statistics tabs output is always csv when i click process data, regardless which output type is choosen. unless I misunderstood how to show data there.

I'm using ubuntu 19.10 with php7.3

Bit/s instead of Bytes

Hi

you are showing the traffic in Bytes per second.
is there any possiblity to change it to Bits per second?

nfdump profiles support

Right now, nfsen-ng is only supporting the live profile. To truly replace the original nfsen, it is needed to support profiles.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    πŸ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. πŸ“ŠπŸ“ˆπŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❀️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.