Comments (3)
sdemchuk
commented on May 24, 2024
I have same issue with IPFIX NAT events logging:
- template in IPFIX flow is present
- tshark or tcpdump correctly decode IPFIX
- nfcapd/nfdump missed some fields (postNATSourceIPv4Address, postNAPTSourceTransportPort, observationTimeMilliseconds)
ipfix.pcap.zip - pcap of IPFIX flow
nfcapd.201705101355.zip - result of nfcapd
#/usr/local/bin/nfcapd -w -p 4739 -T all -E -l /home/sork
Add extension: 2 byte input/output interface index
Add extension: 4 byte input/output interface index
Add extension: 2 byte src/dst AS number
Add extension: 4 byte src/dst AS number
Add extension: dst tos, direction, src/dst mask
Add extension: IPv4 next hop
Add extension: IPv6 next hop
Add extension: IPv4 BGP next IP
Add extension: IPv6 BGP next IP
Add extension: src/dst vlan id
Add extension: 4 byte output packets
Add extension: 8 byte output packets
Add extension: 4 byte output bytes
Add extension: 8 byte output bytes
Add extension: 4 byte aggregated flows
Add extension: 8 byte aggregated flows
Add extension: in src/out dst mac address
Add extension: in dst/out src mac address
Add extension: MPLS Labels
Add extension: IPv4 router IP addr
Add extension: IPv6 router IP addr
Add extension: router ID
Add extension: BGP adjacent prev/next AS
Add extension: time packet received
Add extension: NSEL Common block
Add extension: NSEL xlate ports
Add extension: NSEL xlate IPv4 addr
Add extension: NSEL xlate IPv6 addr
Add extension: NSEL ACL ingress/egress acl ID
Add extension: NSEL username
Add extension: NSEL max username
Add extension: nprobe/nfpcapd latency
Add extension: NEL Common block
Add extension: Compat NEL IPv4
Add extension: NAT Port Block Allocation
Bound to IPv4 host/IP: any, Port: 4739
Startup.
Init IPFIX: Max number of IPFIX tags: 62
Process_ipfix: New exporter: SysID: 1, Observation domain 1 from: 172.16.0.32
Process_ipfix: [1] Add template 256
Process_ipfix: [1] Add template 257
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 1
size = 68
first = 0 [1970-01-01 03:00:00]
last = 0 [1970-01-01 03:00:00]
msec_first = 0
msec_last = 0
src addr = 10.10.1.21
dst addr = 0.0.0.0
src port = 53968
dst port = 0
fwd status = 0
tcp flags = 0x00 ......
proto = 17 UDP
(src)tos = 0
(in)packets = 0
(in)bytes = 0
ip router = 172.16.0.32
received at = 1494413621059 [2017-05-10 13:53:41.059]
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 1
size = 68
first = 0 [1970-01-01 03:00:00]
last = 0 [1970-01-01 03:00:00]
msec_first = 0
msec_last = 0
src addr = 10.10.1.21
dst addr = 0.0.0.0
src port = 63390
dst port = 0
fwd status = 0
tcp flags = 0x00 ......
proto = 17 UDP
(src)tos = 0
(in)packets = 0
(in)bytes = 0
ip router = 172.16.0.32
received at = 1494413621059 [2017-05-10 13:53:41.059]
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 1
size = 68
first = 0 [1970-01-01 03:00:00]
last = 0 [1970-01-01 03:00:00]
msec_first = 0
msec_last = 0
src addr = 10.10.1.21
dst addr = 0.0.0.0
src port = 7070
dst port = 0
fwd status = 0
tcp flags = 0x00 ......
proto = 17 UDP
(src)tos = 0
(in)packets = 0
(in)bytes = 0
ip router = 172.16.0.32
received at = 1494413621059 [2017-05-10 13:53:41.059]
Flow Record:
Flags = 0x06 FLOW, Unsampled
export sysid = 1
size = 68
first = 0 [1970-01-01 03:00:00]
last = 0 [1970-01-01 03:00:00]
msec_first = 0
msec_last = 0
src addr = 10.10.1.21
dst addr = 0.0.0.0
src port = 53989
dst port = 0
fwd status = 0
tcp flags = 0x00 ......
proto = 17 UDP
(src)tos = 0
(in)packets = 0
(in)bytes = 0
ip router = 172.16.0.32
received at = 1494413621059 [2017-05-10 13:53:41.059]
^CFile Block Header:
NumBlocks = 7
Size = 336
id = 2
Ident: 'none' Flows: 4, Packets: 0, Bytes: 0, Sequence Errors: 0, Bad Packets: 0
Total ignored packets: 0
Terminating nfcapd.
# tshark -r ipfix.pcap -V
Frame 1: 98 bytes on wire (784 bits), 98 bytes captured (784 bits) on interface 0
User Datagram Protocol, Src Port: ipfix (4739), Dst Port: ipfix (4739)
Source port: ipfix (4739)
Destination port: ipfix (4739)
Length: 64
Checksum: 0x2f55 [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Cisco NetFlow/IPFIX
Version: 10
Length: 56
Timestamp: May 10, 2017 13:53:29.000000000 EEST
ExportTime: 1494413609
FlowSequence: 531
Observation Domain Id: 1
Set 1
FlowSet Id: Data Template (V10 [IPFIX]) (2)
FlowSet Length: 40
Template (Id = 256, Count = 8)
Template Id: 256
Field Count: 8
Field (1/8): observationTimeMilliseconds
0... .... .... .... = Pen provided: No
.000 0001 0100 0011 = Type: observationTimeMilliseconds (323)
Length: 8
Field (2/8): natEvent
0... .... .... .... = Pen provided: No
.000 0000 1110 0110 = Type: natEvent (230)
Length: 1
Field (3/8): IP_SRC_ADDR
0... .... .... .... = Pen provided: No
.000 0000 0000 1000 = Type: IP_SRC_ADDR (8)
Length: 4
Field (4/8): postNATSourceIPv4Address
0... .... .... .... = Pen provided: No
.000 0000 1110 0001 = Type: postNATSourceIPv4Address (225)
Length: 4
Field (5/8): PROTOCOL
0... .... .... .... = Pen provided: No
.000 0000 0000 0100 = Type: PROTOCOL (4)
Length: 1
Field (6/8): L4_SRC_PORT
0... .... .... .... = Pen provided: No
.000 0000 0000 0111 = Type: L4_SRC_PORT (7)
Length: 2
Field (7/8): postNAPTSourceTransportPort
0... .... .... .... = Pen provided: No
.000 0000 1110 0011 = Type: postNAPTSourceTransportPort (227)
Length: 2
Field (8/8): ingressVRFID
0... .... .... .... = Pen provided: No
.000 0000 1110 1010 = Type: ingressVRFID (234)
Length: 4
Frame 2: 78 bytes on wire (624 bits), 78 bytes captured (624 bits) on interface 0
User Datagram Protocol, Src Port: ipfix (4739), Dst Port: ipfix (4739)
Source port: ipfix (4739)
Destination port: ipfix (4739)
Length: 44
Checksum: 0x3157 [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Cisco NetFlow/IPFIX
Version: 10
Length: 36
Timestamp: May 10, 2017 13:53:34.000000000 EEST
ExportTime: 1494413614
FlowSequence: 531
Observation Domain Id: 1
Set 1
FlowSet Id: Data Template (V10 [IPFIX]) (2)
FlowSet Length: 20
Template (Id = 257, Count = 3)
Template Id: 257
Field Count: 3
Field (1/3): observationTimeMilliseconds
0... .... .... .... = Pen provided: No
.000 0001 0100 0011 = Type: observationTimeMilliseconds (323)
Length: 8
Field (2/3): natEvent
0... .... .... .... = Pen provided: No
.000 0000 1110 0110 = Type: natEvent (230)
Length: 1
Field (3/3): Unknown(283)
0... .... .... .... = Pen provided: No
.000 0001 0001 1011 = Type: Unknown (283)
Length: 4
Frame 3: 166 bytes on wire (1328 bits), 166 bytes captured (1328 bits) on interface 0
User Datagram Protocol, Src Port: ipfix (4739), Dst Port: ipfix (4739)
Source port: ipfix (4739)
Destination port: ipfix (4739)
Length: 132
Checksum: 0xd7b1 [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
Cisco NetFlow/IPFIX
Version: 10
Length: 124
Timestamp: May 10, 2017 13:53:40.000000000 EEST
ExportTime: 1494413620
FlowSequence: 531
Observation Domain Id: 1
Set 1
FlowSet Id: (Data) (256)
FlowSet Length: 108
Flow 1
Observation Time Milliseconds: May 10, 2017 13:53:41.545000000 EEST
Nat Event: 5
SrcAddr: 10.10.1.21 (10.10.1.21)
Post NAT Source IPv4 Address: 31.134.121.0 (31.134.121.0)
Protocol: 17
SrcPort: 53968
Post NAPT Source Transport Port: 15601
Ingress VRFID: 0
Flow 2
Observation Time Milliseconds: May 10, 2017 13:53:41.545000000 EEST
Nat Event: 4
SrcAddr: 10.10.1.21 (10.10.1.21)
Post NAT Source IPv4 Address: 31.134.121.0 (31.134.121.0)
Protocol: 17
SrcPort: 63390
Post NAPT Source Transport Port: 44837
Ingress VRFID: 0
Flow 3
Observation Time Milliseconds: May 10, 2017 13:53:42.056000000 EEST
Nat Event: 5
SrcAddr: 10.10.1.21 (10.10.1.21)
Post NAT Source IPv4 Address: 31.134.121.0 (31.134.121.0)
Protocol: 17
SrcPort: 7070
Post NAPT Source Transport Port: 49517
Ingress VRFID: 0
Flow 4
Observation Time Milliseconds: May 10, 2017 13:53:42.056000000 EEST
Nat Event: 4
SrcAddr: 10.10.1.21 (10.10.1.21)
Post NAT Source IPv4 Address: 31.134.121.0 (31.134.121.0)
Protocol: 17
SrcPort: 53989
Post NAPT Source Transport Port: 38464
Ingress VRFID: 0
from nfdump.
bbayles
commented on May 24, 2024
@sdemchuk, you may want to check out the SiLK tools; I've found them to be more compatible with IPSEC when I had this problem with nfdump.
(apologies in advance; I don't mean to be rude in pointing someone to another project)
from nfdump.
phaag
commented on May 24, 2024
@gabalino the message
Process_ipfix: [0] option template length error: size left 20 too small for 5 scopes length and 1 options length
obviously means, that your exporter sends corrupted messages. I would need to have more information about this device and a pcap would help.
Most compile warnings have been fixed.
@sdemchuk
I don't consider your issue the same. As of now, nfdump does not support events sent in IPFIX packets. This might be supported in future. In any case I would need the device you are using.
from nfdump.
Related Issues (20)
- sfcapd -T Extensions 1.6.x missing in 1.7.x HOT 2
- nfprofile: Skip unknown record type 13 (after upgrrading from 1.6.20 to 1.7.3) HOT 8
- Sfcapd not processing netflow... HOT 2
- Include dependencies? HOT 4
- sfcapd not working properly after last commits HOT 4
- GCC14 build failure HOT 5
- nfdump: Skip unknown record type 9 HOT 6
- Troubleshooting NetFlow Data Collection and Router Address Display HOT 4
- Support for NetFlow version 10 HOT 2
- Decreased nfdump performance after upgrading from 1.6.17 HOT 2
- sfcapd not acknowledging -W <Worker> HOT 3
- IPFIX (V10) support for dot1q VLAN IDs HOT 23
- Typo in IPFIX fields HOT 1
- nfdump default workers calculation on low-cpu systems HOT 1
- AS Information HOT 1
- PBlock variables are not printing properly with custom format.
- Bad magic 0x10CF HOT 4
- Runnng nfcapd/sfcapd simultaneously in WSL segfaults... HOT 8
- issues after upgrade - lz4 compression stopped working HOT 4
- nfdump - help/error text for compression argument HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nfdump.