Comments (4)
I didn't look deep in the issue, but what you describe in "expected behaviour" is not what is actually expected.
An event handler in a subscription should execute exactly once for each event
Eventuous never aimed to provide any guarantee for "exactly once" event processing. Doing so would require that the checkpoint is stored in the same database where the subscription writes to (if it does) and it must happen in the same transaction. Checkpointing and event processing aren't coupled in Eventuous. so it's physically impossible.
This is from the docs:
When using Eventuous, it's important to remember that it does not enforce exactly one event processing rule (although it can), as it would have a negative impact on the subscription's performance. Therefore, when the handler has processed an event, it might eventually need to process it again when the application restarts after a crash. It might sound a bit scary, but in reality, those events will be delivered again in the same order, and it's easy to mitigate the issue by ensuring that the projection handler is idempotent. For our example, we could do that by using updateOne operation with the option isUpsert set to true instead of using the insertOne operation. Any update operation is by definition idempotent as long as it doesn't use operations on the existing state like inc or dec. That's why it's essential to only use event properties in updates, so the event needs to contain enough information for the projecting handler to execute the update without using the current projected document state.
I agree, though, that event handler being called twice for each event is weird. But, the tests I have don't exhibit such behaviour. I need to look closer at your repro to understand what is going on.
from eventuous.
I understand about CAP and all of that, but that shouldn't really be applicable in this simple example. This is a case of just one subscription and a single local SQL Server instance for both messages and checkpoints. There isn't any contention of the sort that would be expected to lead to message delivery being consistently retried.
Since I posted the original code, I've discovered that the second invocation of the event handler is occurring while the first invocation is still in progress, so it's some kind of weird race. That also explains why some of my first attempts at resolving the problem were not successful.
On a hunch, I tried rewriting the event handler as a sync rather than async function, and that fixes the problem, which further confirms that there is some kind of async-related race. I've updated the code to simplify some things (no more DI required), and there are two branches, sync-read
and async-read
, that show the correct and incorrect behavior, respectively.
from eventuous.
Yeah, I don't disagree, don't get me wrong. I was thinking about it last night. Maybe it's related to the polling query back pressure that I added to 0.15. I got to think that the current test suite runs like "produce 100 events, then consume 100 events", where if you produce one and consume one, then do it again, there might be something there (checkpoint basically).
I will add a test like that to see if it can be reproduced internally. Thanks for reporting :)
from eventuous.
Should this be closed? 4a65311 seems to have fixed the problem.
from eventuous.
Related Issues (20)
- Ability to return a custom return type for a command API reponse HOT 5
- SQL Server __schema__.Streams.StreamName is too big to be indexed
- Unable to create isolated Aggregates with State and Id HOT 5
- SQL Server Subscription skips an event on non-initial startup
- Provide StreamContext in UpdateBuilder<TBuilder>.Configure() for MondoDb HOT 1
- Health check stays "healthy" even if Event Handler throws an exception HOT 5
- Transient error drops the subscription. (Connection reset by peer) HOT 6
- Add subscription drop/resubscribe tests HOT 1
- Add subscription measure for RDBMS subs HOT 1
- Error at schema initialization (PostgresException: '42704: type public.stream_message[] does not exist) HOT 1
- Eventuous should always return `Result<TState>` HOT 2
- Does it make sense to keep `Aggregate` without state? HOT 9
- Preserve original exception stacktrace in command service HOT 1
- Enable custom configuration of NpgsqlDataSourceBuilder when using AddEventuousPostgres HOT 1
- DuplicateTypeException in CommandService HOT 11
- "Can not start an Activity that was already started" in BaseProducer
- Memory leak (CancellationTokenSource) in AsyncHandlingFilter HOT 1
- Use source generator for type mapping HOT 1
- Use source generator for mapping HTTP API to command handlers HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from eventuous.