Comments (9)
phaag
commented on September 23, 2024
1
The patch fixes the uninitialised memory reference.
from nfdump.
aldem
commented on September 23, 2024
Update: similar behavior is observed with LZO, so this really has nothing to do with compression code itself:
==362798== Conditional jump or move depends on uninitialised value(s)
==362798== at 0x4853444: lzo1x_1_compress_core (minilzo.c:5153)
==362798== by 0x4852B7A: lzo1x_1_compress (minilzo.c:5234)
==362798== by 0x4865C51: Compress_Block_LZO (nffile.c:201)
==362798== by 0x4865C51: nfwrite (nffile.c:1239)
==362798== by 0x4864EDE: nfwriter (nffile.c:1291)
==362798== by 0x48E0EA6: start_thread (pthread_create.c:477)
==362798== by 0x4AF1A2E: clone (clone.S:95)
and BZ2:
==362946== Conditional jump or move depends on uninitialised value(s)
==362946== at 0x491126D: ??? (in /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4)
==362946== by 0x491186A: BZ2_bzCompress (in /usr/lib/x86_64-linux-gnu/libbz2.so.1.0.4)
==362946== by 0x4865CCC: Compress_Block_BZ2 (nffile.c:304)
==362946== by 0x4865CCC: nfwrite (nffile.c:1247)
==362946== by 0x4864EDE: nfwriter (nffile.c:1291)
==362946== by 0x48E0EA6: start_thread (pthread_create.c:477)
==362946== by 0x4AF1A2E: clone (clone.S:95)
from nfdump.
phaag
commented on September 23, 2024
Can you please specify the exact command line including valgrind.
from nfdump.
aldem
commented on September 23, 2024
Something like this:
valgrind --tool=memcheck -- /opt/nfdump/bin/nfdump -r nfcapd.20230304233310 -w nfcapd.20230304233310.agg -a -z
from nfdump.
phaag
commented on September 23, 2024
Thanks! - valgrind could at least be a bit more precise ... the output is a bit of limited value.
from nfdump.
aldem
commented on September 23, 2024
Better this to track origin:
valgrind --tool=memcheck --track-origins=yes ...
Just found a reference:
==364794== Uninitialised value was created by a stack allocation
==364794== at 0x40EF40: AddFlowCache (nflowcache.c:968)
and this:
==364794== Syscall param write(buf) points to uninitialised byte(s)
==364794== at 0x48EAFEF: __libc_write (write.c:26)
==364794== by 0x48EAFEF: write (write.c:24)
==364794== by 0x4865E17: nfwrite (nffile.c:1266)
==364794== by 0x4864EDE: nfwriter (nffile.c:1291)
==364794== by 0x48E0EA6: start_thread (pthread_create.c:477)
==364794== by 0x4AF1A2E: clone (clone.S:95)
==364794== Address 0x13afe044 is 4 bytes inside a block of size 10,485,760 alloc'd
==364794== at 0x483877F: malloc (vg_replace_malloc.c:307)
==364794== by 0x4864DA2: NewFile (nffile.c:567)
==364794== by 0x48649D3: OpenNewFile (nffile.c:782)
==364794== by 0x4044CE: main (nfdump.c:1106)
==364794== Uninitialised value was created by a stack allocation
==364794== at 0x40EF40: AddFlowCache (nflowcache.c:968)
Most likely there is some corruption introduced in either AddFlowCache or in arguments to AddFlowCache.
from nfdump.
aldem
commented on September 23, 2024
Looks like there is really some out-of-bounds memory access, even without compression involved (only -a):
==378232== 28 errors in context 1 of 1:
==378232== Syscall param write(buf) points to uninitialised byte(s)
==378232== at 0x48EEFEF: __libc_write (write.c:26)
==378232== by 0x48EEFEF: write (write.c:24)
==378232== by 0x4869A97: nfwrite (nffile.c:1265)
==378232== by 0x4868B6E: nfwriter (nffile.c:1290)
==378232== by 0x48E4EA6: start_thread (pthread_create.c:477)
==378232== by 0x4AF5A2E: clone (clone.S:95)
==378232== Address 0xf6fb0d2 is 146 bytes inside a block of size 10,485,760 alloc'd
==378232== at 0x483877F: malloc (vg_replace_malloc.c:307)
==378232== by 0x486897E: NewFile (nffile.c:567)
==378232== by 0x4868663: OpenNewFile (nffile.c:782)
==378232== by 0x4044CE: main (nfdump.c:1106)
I have to learn the code first to find it out, without understanding how it does things (supposed to do at least) it is hard - it could be anywhere but for sure somewhere in code related to aggregation.
from nfdump.
phaag
commented on September 23, 2024
I will try to dive into that next week. I need a bit more time.
from nfdump.
phaag
commented on September 23, 2024
Just to add a comment oh this: It was not an out-of-bounds memory access. The outFlags variable, which is really seldom used, was not properly initialised when doing aggregation. Valgrind barks at the first reading event of that memory location. This access is no earlier than storing the data in the file, or before that - compressing the data. However, you can force this also by fmt printing the flows instead storing to disk.
from nfdump.
Related Issues (20)
- Event time when compiled with NSEL/NEL option.
- Aggregation and ICMP proto HOT 4
- make errors due to #423? HOT 1
- Aggregated flows consume more space than collected and other strange things with 1.7 HOT 14
- geolookup truncate autonomous_system_organization name HOT 5
- Destination Location header in formated output is called "Src IP location info" HOT 1
- Feature Request: autonomous_system_organization name in nfdump output HOT 7
- nftrack [regression]: -r <file> aborts if <file> is not in current directory HOT 1
- Nfcapd's packet repeater doesn't seem to work HOT 2
- Memory leak in nfcapd HOT 3
- How to capture the ASN of Netflow HOT 10
- Unidentified AS 0 HOT 2
- Nfdump collecting interval HOT 2
- Nfanon and filter mode? HOT 2
- request/ selective anonymization HOT 3
- request: nfreplay lose exporter IP info HOT 6
- tiny build issue in make: ../../ylwrap: line 176: yacc: command not found HOT 2
- missing nfcapd instance for device HOT 1
- ZSTD compression HOT 34
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nfdump.