Comments (7)
Hey @ssmails ,
I don't have any idea how it could happen . You mean you received messages more than once ?
could you please provide a unit test? Or a code snippet that show it is happening?
from mockafka-py.
@ssmails please look at this PR
I think you forgot to add clean=True
to your admin client
in this test, I added parameterized test case that checks the number of produced messages with the number of consumed message
from mockafka-py.
@alm0ra thanks for looking into this. I copied the test sample you had and added a print msg as part of the test after the consumer runs, and am able to reproduce the issue I mentioned.
I reduced the dataset to 4 runs as well, so its more clear.
In run 2 below, you see the 2 messages are the same. They should be different since a uuid is used as the msg produced per https://github.com/alm0ra/mockafka-py/pull/62/files#diff-6d0ff70a7ffbc6670cd1e7d92364fa4ad5c3e155734668e5f668aee512b59e8aR62
test_consumer_consistency.py::test_consumer_consistency[2] PASSED [ 50%]
recv: {'u': '34635fbb-bdf3-44d3-baf0-e5fb4681fc53'}
recv: {'u': '34635fbb-bdf3-44d3-baf0-e5fb4681fc53'}
Similarly in test 3 and 4 below, we see that sometimes messages are duplicates, and sometimes unique.
If I run the test multiple times, everytime, the behaviour is different.
============================= test session starts ==============================
collecting ... collected 4 items
test_consumer_consistency.py::test_consumer_consistency[1] PASSED [ 25%]
recv: {'u': '01d35b5b-b7da-4975-b91c-35f1d8bc8ff4'}
test_consumer_consistency.py::test_consumer_consistency[2] PASSED [ 50%]
recv: {'u': '34635fbb-bdf3-44d3-baf0-e5fb4681fc53'}
recv: {'u': '34635fbb-bdf3-44d3-baf0-e5fb4681fc53'}
test_consumer_consistency.py::test_consumer_consistency[3] PASSED [ 75%]
recv: {'u': '111f5a57-9815-4f16-b1d1-ab5e42af5fe9'}
recv: {'u': '111f5a57-9815-4f16-b1d1-ab5e42af5fe9'}
recv: {'u': 'd6b99d87-9636-4f76-a351-8c8b98abe45f'}
test_consumer_consistency.py::test_consumer_consistency[4] PASSED [100%]
recv: {'u': 'db027dff-c5a5-4a85-986f-e7007e9d3efe'}
recv: {'u': '2919cdc3-5570-475a-a647-6b0327e011f7'}
recv: {'u': '2919cdc3-5570-475a-a647-6b0327e011f7'}
recv: {'u': '1afacb8e-bc2f-435f-89ce-46d5b568e1a6'}
============================== 4 passed in 0.05s ===============================
Process finished with exit code 0
from mockafka-py.
@ssmails
thanks for your raising the issue.
i found it by adding
values_list = [item["u"] for item in messages]
assert len(messages) == len(set(values_list))
to my test
from mockafka-py.
Dear @ssmails
I have identified the problem. As outlined in the pull request (PR), we neglected to implement single message commit functionality. Consequently, I refrained from committing anything when you passed this parameter. However, I have now included a new assertion to ensure that we do not consume a message more than once after committing. Once the PR is merged, the issue will be resolved. Should you encounter any further problems, please do not hesitate to inform me, and I will be glad to assist.
from mockafka-py.
Fixed in this release-v0.1.54
from mockafka-py.
Thanks @alm0ra , will check.
from mockafka-py.
Related Issues (20)
- Async support? HOT 2
- add help for aiokafka support in readme
- `send_and_wait` doesn't properly mock `aiokafka.send_and_wait` HOT 4
- FakeConsumer consume does not match confluent kafka API HOT 2
- Add more test coverage and Edge case
- Problem in commit single message
- Error handling in CustomDict HOT 4
- Message timestamp interface doesn't match confluent_kafka HOT 2
- Type hints cause TypeError under Python 3.8 HOT 4
- `FakeAIOKafkaProducer` need `stop` function HOT 1
- Missing 0.1.56 pypi release HOT 1
- Remove unused `parameterized` dependency from package
- Package docs link is out of date
- `FakeAIOKafkaConsumer.subscribe` missing arguments
- Message class incompatible with aiokafka's ConsumerRecord HOT 1
- `FakeAIOKafkaConsumer.getone` returns `None` but should block HOT 5
- `FakeAIOKafkaConsumer` never raises error about being closed
- Generic and unsolicited emails are not a good way to bring attention to your project HOT 4
- FakeAIOKafkaConsumer subscription returns list instead of frozenset HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mockafka-py.