Bookmarking jumps over one record about py-idstools HOT 3 CLOSED

You are correct, I ran into this earlier this week. It is fixed in my development branch, "hack" but I will be rolling it into master today probably and doing a new release to pypi.

The problem is that SpoolEventReader is abstracted enough away from the underlying file that you can't use to current offset for bookmarking, as its the following "event" record that triggers all the queued records to be turned into an event. Then you bookmark the current file offset which is one record into the next event. On next start it will skip forward to the start of the next event.

To properly fix this I had to move bookmarking into SpoolEventReader, as well as the delete option. These are both available in the "hack" branch.. But will go to master probably today.

from py-idstools.

Wow that was fast, thank you very much! I am going to wait until you release a new release and complete the other stuff (please don't hurry because of me).

from py-idstools.

I tagged the 0.4.0 release. You'll notice that SpoolEventReader has less parameters. You just tell it to bookmark and delete if needed and it takes care of it for you.

I may have been a little short sighted. It still may make sense for the caller to do the bookmarking in some uses cases, so I'll probably re-add some arguments soon. It was just easier to reason about with less options.

from py-idstools.