Giter Club home page Giter Club logo

Comments (4)

alexeyzimarev avatar alexeyzimarev commented on September 25, 2024

I didn't look deep in the issue, but what you describe in "expected behaviour" is not what is actually expected.

An event handler in a subscription should execute exactly once for each event

Eventuous never aimed to provide any guarantee for "exactly once" event processing. Doing so would require that the checkpoint is stored in the same database where the subscription writes to (if it does) and it must happen in the same transaction. Checkpointing and event processing aren't coupled in Eventuous. so it's physically impossible.

This is from the docs:

When using Eventuous, it's important to remember that it does not enforce exactly one event processing rule (although it can), as it would have a negative impact on the subscription's performance. Therefore, when the handler has processed an event, it might eventually need to process it again when the application restarts after a crash. It might sound a bit scary, but in reality, those events will be delivered again in the same order, and it's easy to mitigate the issue by ensuring that the projection handler is idempotent. For our example, we could do that by using updateOne operation with the option isUpsert set to true instead of using the insertOne operation. Any update operation is by definition idempotent as long as it doesn't use operations on the existing state like inc or dec. That's why it's essential to only use event properties in updates, so the event needs to contain enough information for the projecting handler to execute the update without using the current projected document state.

I agree, though, that event handler being called twice for each event is weird. But, the tests I have don't exhibit such behaviour. I need to look closer at your repro to understand what is going on.

from eventuous.

Steve-OH avatar Steve-OH commented on September 25, 2024

I understand about CAP and all of that, but that shouldn't really be applicable in this simple example. This is a case of just one subscription and a single local SQL Server instance for both messages and checkpoints. There isn't any contention of the sort that would be expected to lead to message delivery being consistently retried.

Since I posted the original code, I've discovered that the second invocation of the event handler is occurring while the first invocation is still in progress, so it's some kind of weird race. That also explains why some of my first attempts at resolving the problem were not successful.

On a hunch, I tried rewriting the event handler as a sync rather than async function, and that fixes the problem, which further confirms that there is some kind of async-related race. I've updated the code to simplify some things (no more DI required), and there are two branches, sync-read and async-read, that show the correct and incorrect behavior, respectively.

from eventuous.

alexeyzimarev avatar alexeyzimarev commented on September 25, 2024

Yeah, I don't disagree, don't get me wrong. I was thinking about it last night. Maybe it's related to the polling query back pressure that I added to 0.15. I got to think that the current test suite runs like "produce 100 events, then consume 100 events", where if you produce one and consume one, then do it again, there might be something there (checkpoint basically).

I will add a test like that to see if it can be reproduced internally. Thanks for reporting :)

from eventuous.

Steve-OH avatar Steve-OH commented on September 25, 2024

Should this be closed? 4a65311 seems to have fixed the problem.

from eventuous.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.