issue with u2eve about py-idstools HOT 14 CLOSED

Thanks. I'm still not sure why this is happening, but will look into it further but it may be a couple days.

Any plan for that evebox can directly read unified files?

I've been thinking about this recently. As I already have unified2 and rule parsers written in Go it wouldn't actually take much time. Perhaps I'll consider it more seriously, but won't be able to act on it for at least a week.

from py-idstools.

Can you tell me how you are running u2eve? The idstools source distribution contains a file, tests/merged.log that contains 6 events. u2spewfoo prints 6 event records along with some packet records. u2eve also prints 6 records. I admit thats a rather simplistic test, but I need more information in order to replicate.

from py-idstools.

Snort created a new spool file with only one entry. u2eve did not recognise this. I've triggered a test command (nmap) and snort created the second event. This time u2eve reports the prior event, but not the last (nmap) event. It does no matter if u2eve run's in follow mode or as single command. I've made a copy of the snort spool file and can provide it, if you need it. u2eve run's with --snort-conf /etc/snort/snort.conf --directory /var/log/snort --prefix snort.u2 --output /var/log/snort/eve.json --bookmark --follow

from py-idstools.

Yes, please provide. u2eve may present some delay due to the formatting of unified2 files. It will read an event record, then wait for a following packet record before outputting the event. This is so it can present the alert/event and the packet in a single record. If there is no following packet, the event won't be output until the next event. But I have test cases for this.

Does your unified output contain packets?

from py-idstools.

Yes, the unified log contain also packets. Meanwhile i've made a other test and removed the bookmark and u2eve output file. This time u2eve reports all events. I think it is a problem with the bookmark thing. I forgot to backup the bookmark file, so i think it is senseless to provide the snort file without the bookmark file.

from py-idstools.

I've switched back to follow mode with bookmark file, and the u2eve output file looks like malformed. The last event appear two times and the last entry miss some information (event-type and following until packets).I used the u2eve output file as input for evebox. Any plan for that evebox can directly read unified files?
debug.gz
.

from py-idstools.

Any chance you can share your unified2 file? Privately if needed - [email protected]

from py-idstools.

Oh, and what does your "output unified2" look like in your snort.conf? I may have made some assumptions that don't fit all use cases.

from py-idstools.

Snort run with the default settings. output unified2: filename snort.u2, limit 128

from py-idstools.

I've pushed an update to git master that may help out your situation. While I was unable to replicate corrupt JSON, I was able to create incomplete records, which led to an erroneous record after restart. So I'm curious if it would help your use case as well.

from py-idstools.

Any update here? I'd like to tag a release soon and would like to mark this as fixed.

Thanks.

from py-idstools.

Sorry, busy week. So far it looks good. I did not see any malformed json entries and currently all snort events are reported correct. I'm suprised about the --follow command. u2eve now always run in continuous mode.

from py-idstools.

I'm suprised about the --follow command. u2eve now always run in continuous mode.

Oops, fixing.

from py-idstools.

See 8200f4c

Closing.

from py-idstools.