Comments (5)
noa-appletech
commented on May 24, 2024
It seems to me that this issue is related to:
https://sourceforge.net/p/nfdump/mailman/message/31901489/
but in this case we do have a source address; however, it seems it is not properly read by nfcapd.
from nfdump.
noa-appletech
commented on May 24, 2024
I add below the template packet (decoded by Wireshark and exported as text) for reference:
No. Time Source Destination Protocol Length Info
877 2016-07-31 00:23:44.691830 195.251.204.254 195.251.204.212 CFLOW 163 total: 2 (v9) records Obs-Domain-ID= 0 [Data:257] [Data-Template:257]
Frame 877: 163 bytes on wire (1304 bits), 163 bytes captured (1304 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Jul 31, 2016 00:23:44.691830000 GTB Daylight Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1469913824.691830000 seconds
[Time delta from previous captured frame: 0.126154000 seconds]
[Time delta from previous displayed frame: 0.126154000 seconds]
[Time since reference or first frame: 401.126018000 seconds]
Frame Number: 877
Frame Length: 163 bytes (1304 bits)
Capture Length: 163 bytes (1304 bits)
[Frame is marked: True]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:udp:cflow]
[Coloring Rule Name: UDP]
[Coloring Rule String: udp]
Ethernet II, Src: CiscoInc_52:38:11 (f4:0f:1b:52:38:11), Dst: DigitalE_2e:f5:53 (aa:00:00:2e:f5:53)
Destination: DigitalE_2e:f5:53 (aa:00:00:2e:f5:53)
Address: DigitalE_2e:f5:53 (aa:00:00:2e:f5:53)
.... ..1. .... .... .... .... = LG bit: Locally administered address (this is NOT the factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: CiscoInc_52:38:11 (f4:0f:1b:52:38:11)
Address: CiscoInc_52:38:11 (f4:0f:1b:52:38:11)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: 195.251.204.254, Dst: 195.251.204.212
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 149
Identification: 0x6ebf (28351)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 0
Time to live: 255
Protocol: UDP (17)
Header checksum: 0x2ace [validation disabled]
[Good: False]
[Bad: False]
Source: 195.251.204.254
Destination: 195.251.204.212
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
User Datagram Protocol, Src Port: 57095 (57095), Dst Port: 9995 (9995)
Source Port: 57095
Destination Port: 9995
Length: 129
Checksum: 0x9d37 [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
[Stream index: 1]
Cisco NetFlow/IPFIX
Version: 9
Count: 2
SysUptime: 146664.635723936 seconds
Timestamp: Jul 31, 2016 00:23:44.000000000 GTB Daylight Time
CurrentSecs: 1469913824
FlowSequence: 59948 (expected 271514)
[Expert Info (Warn/Sequence): Unexpected flow sequence for domain ID 0 (expected 271514, got 59948)]
[Unexpected flow sequence for domain ID 0 (expected 271514, got 59948)]
[Severity level: Warn]
[Group: Sequence]
SourceId: 0
FlowSet 1 [id=257] (1 flows)
FlowSet Id: (Data) (257)
FlowSet Length: 57
[Template Frame: 877]
Flow 1
DstAddr: 2001:648:2011:10::234
Protocol: TCP (6)
SrcPort: 46042 (46042)
DstPort: 80 (80)
Octets: 495
Packets: 5
[Duration: 0.012000000 seconds (switched)]
StartTime: 146647.752000000 seconds
EndTime: 146647.764000000 seconds
SrcAddr: 2001:648:2011:8010::211
FlowSet 2 [id=0] (Data Template): 257
FlowSet Id: Data Template (V9) (0)
FlowSet Length: 44
Template (Id = 257, Count = 9)
Template Id: 257
Field Count: 9
Field (1/9): IPV6_DST_ADDR
Type: IPV6_DST_ADDR (28)
Length: 16
Field (2/9): PROTOCOL
Type: PROTOCOL (4)
Length: 1
Field (3/9): L4_SRC_PORT
Type: L4_SRC_PORT (7)
Length: 2
Field (4/9): L4_DST_PORT
Type: L4_DST_PORT (11)
Length: 2
Field (5/9): BYTES
Type: BYTES (1)
Length: 4
Field (6/9): PKTS
Type: PKTS (2)
Length: 4
Field (7/9): FIRST_SWITCHED
Type: FIRST_SWITCHED (22)
Length: 4
Field (8/9): LAST_SWITCHED
Type: LAST_SWITCHED (21)
Length: 4
Field (9/9): IPV6_SRC_ADDR
Type: IPV6_SRC_ADDR (27)
Length: 16
[Expected Sequence Number: 271514]
[Previous Frame in Sequence: 876]
Nick
from nfdump.
noa-appletech
commented on May 24, 2024
Note: I had faced and reported (to nfsen-discuss mailing list) the same issue a year ago, but at the time I did not identify the source of the problem (misinterpretation of IPv6 traffic).
Ref.: https://sourceforge.net/p/nfsen/mailman/message/34329416/
At that time the issue was not investigated further to trace the piece of nfdump code that causes it.
I do hope this time it will be finally resolved in nfdump source code, since it's an nfdump issue.
from nfdump.
phaag
commented on May 24, 2024
I analyzed the packet capture, which Nick provided. It turned out, that the exporter sends buggy templates for IPv6. Only 1 out of ~30 template refreshes are correct IPv6, but the majority are buggy:
buggy templates - contain IPv4 records:
[0] Template ID: 257
template size: 80 buffersize: 80
found extension 0 for type: 21(time sec end), at index: 26, input length: 4 output length: 4 Extension: 0, Offset: 0
found extension 0 for type: 22(time sec create), at index: 27, input length: 4 output length: 4 Extension: 0, Offset: 4
found extension 0 for type: 1(bytes), at index: 1, input length: 4 output length: 8 Extension: 0, Offset: 8
found extension 0 for type: 2(packets), at index: 3, input length: 4 output length: 8 Extension: 0, Offset: 12
found extension 4 for type: 10(input SNMP), at index: 13, input length: 2 output length: 2 Extension: 4, Offset: 16
Enable extension: 4: 2 byte input/output interface index
found extension 4 for type: 14(output SNMP), at index: 18, input length: 2 output length: 2 Extension: 4, Offset: 18
found extension 0 for type: 8(V4 src addr), at index: 11, input length: 4 output length: 4 Extension: 0, Offset: 20
found extension 0 for type: 12(V4 dst addr), at index: 16, input length: 4 output length: 4 Extension: 0, Offset: 24
found extension 0 for type: 4(proto), at index: 7, input length: 1 output length: 1 Extension: 0, Offset: 28
found extension 0 for type: 5(tos), at index: 8, input length: 1 output length: 1 Extension: 0, Offset: 29
found extension 0 for type: 7(src port), at index: 10, input length: 2 output length: 2 Extension: 0, Offset: 30
found extension 0 for type: 11(dst port), at index: 15, input length: 2 output length: 2 Extension: 0, Offset: 32
found extension 0 for type: 48(sampler ID), at index: 44, input length: 1 output length: 1 Extension: 0, Offset: 34
Skip unknown element type: 51, Length: 1
found extension 9 for type: 15(V4 next hop IP), at index: 20, input length: 4 output length: 4 Extension: 9, Offset: 36
Enable extension: 9: IPv4 next hop
found extension 8 for type: 13(V4 dst mask), at index: 17, input length: 1 output length: 1 Extension: 8, Offset: 40
Enable extension: 8: dst tos, direction, src/dst mask
found extension 8 for type: 9(V4 src mask), at index: 12, input length: 1 output length: 1 Extension: 8, Offset: 41
found extension 0 for type: 6(flags), at index: 9, input length: 1 output length: 1 Extension: 0, Offset: 42
correct records: contain IPv6 records:
[0] Template ID: 257
template size: 40 buffersize: 40
found extension 0 for type: 28(V6 dst addr), at index: 35, input length: 16 output length: 16 Extension: 0, Offset: 0
found extension 0 for type: 4(proto), at index: 7, input length: 1 output length: 1 Extension: 0, Offset: 16
found extension 0 for type: 7(src port), at index: 10, input length: 2 output length: 2 Extension: 0, Offset: 17
found extension 0 for type: 11(dst port), at index: 15, input length: 2 output length: 2 Extension: 0, Offset: 19
found extension 0 for type: 1(bytes), at index: 1, input length: 4 output length: 8 Extension: 0, Offset: 21
found extension 0 for type: 2(packets), at index: 3, input length: 4 output length: 8 Extension: 0, Offset: 25
found extension 0 for type: 22(time sec create), at index: 27, input length: 4 output length: 4 Extension: 0, Offset: 29
found extension 0 for type: 21(time sec end), at index: 26, input length: 4 output length: 4 Extension: 0, Offset: 33
found extension 0 for type: 27(V6 src addr), at index: 34, input length: 16 output length: 16 Extension: 0, Offset: 37
The data stream sent by the exporter always decodes data according to the IPv6 template, but announces IPv4. Therefore most IPv6 flows end up as garbage.
from nfdump.
phaag
commented on May 24, 2024
Issue closed - not an nfdump issue. Exporter related config or bug. Analyzed files are reported correctly
from nfdump.
Related Issues (20)
- Can't find ftlib.h durning configure ft2nfdump HOT 2
- RAM consumption HOT 2
- NEL Port Block Allocation / Deallocation Events HOT 1
- Is it possible to know if a flow contained fragmented traffic? HOT 6
- nfdump current (1.7.3) has a bug exporting NSEL (cisco ASA) fw events HOT 3
- when daemonizing, requesting to set uid and gid to some user AND writing PIDfile -> permission denied encountered HOT 4
- feature: it will be very cool if nfcapd switch '-n' allow specifying port to listen to. not globally single '-p' but per-configured exporter HOT 3
- sfcapd -T Extensions 1.6.x missing in 1.7.x HOT 2
- nfprofile: Skip unknown record type 13 (after upgrrading from 1.6.20 to 1.7.3) HOT 8
- Sfcapd not processing netflow... HOT 2
- Include dependencies? HOT 4
- sfcapd not working properly after last commits HOT 4
- GCC14 build failure HOT 5
- nfdump: Skip unknown record type 9 HOT 6
- Troubleshooting NetFlow Data Collection and Router Address Display HOT 4
- Support for NetFlow version 10 HOT 2
- Decreased nfdump performance after upgrading from 1.6.17 HOT 2
- sfcapd not acknowledging -W <Worker> HOT 3
- IPFIX (V10) support for dot1q VLAN IDs HOT 23
- Typo in IPFIX fields HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nfdump.