pyeventsourcing / esdbclient Goto Github PK

@ylorph, as requested here's a GH Issue to discuss the inclusive/exclusive behaviour of EventStoreDB for subscriptions from a specific commit position. As we saw, catch up subscriptions seem to be definitely exclusive, but persistent subscriptions less than definitely exclusive. I will post a link to the test for this shortly...

I'm new to python, and I'm using this library for IPC calls to other non-python programs.
However, unexpectedly, the event published from this library can't be read as same as others published on NodeJS client.

After a while I encountered this problem, I'd changed contentType from application/octet-stream to application/json and this kind of fix make it works anyway. I'm not good at eventstoreDB, but I guess this is related to event types, such as JSONEventType or BinaryEventType in node client.

I'm not sure but it seems application/json is more recommended way to publish.
Is there any reason to use application/octet-stream? or Do you have any plans for event type support?

I'm using event store consumers in our system as a data source for the read model. How much can I trust the event store as a message broker? That some events will reach listeners with a guarantee without loss.

I've noticed that catchup subscriptions support resolve_links=True, so we can listen to event type projection streams, and read the events from them

Persistent subscriptions, however, do not have this capability, and in 1.0.2 will return a link instead of the resolved event

Please add the resolve_links option to the read_subscription_to_stream method

NOTE: .NET client supports this behaviour so the server should support it

This year, work was done on the server to support 'caught up' and 'fell behind' notifications for catch-up subscriptions. It would be great if this client supported those messages.

Hi.
I assumed that persistent subscription events that were ACK, should not be repeated to consumers anymore. But it appears that they are. Here tests that replicate this scenario.

Is this a valid assumption, or I assume the wrong behavior?

It's quite minor issue, but I think adding this for published version is useful.

v0.5
v0.6

Would you mind adding some tags for commits for these versions?
Thanks.

version = "1.0.10"

After updating the library version to later, I got this error, before that code was working correctly, I didn't touch anything.
To reproduce need to run the consumer via the management command. In a simply consumer without the django command also raised the error.

class Command(BaseCommand):
    help = "Run event store consumers"

    def handle(self, *args, **options) -> None:
        client = get_event_store_client()

        subscription = client.read_subscription_to_stream(
            group_name="my_group",
            stream_name="$ce-acquired_orders",
        )

        for event in subscription:
            # Acknowledge the received event.
            subscription.ack(event)

Traceback (most recent call last):
  File "/.venv/lib/python3.11/site-packages/grpc/_channel.py", line 274, in consume_request_iterator
    request = next(request_iterator)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/.venv/lib/python3.11/site-packages/esdbclient/persistent.py", line 514, in __next__
    event_id, action = self._ack_queue.get(timeout=get_timeout)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.6/Frameworks/Python.framework/Versions/3.11/lib/python3.11/queue.py", line 173, in get
    raise ValueError("'timeout' must be a non-negative number")
ValueError: 'timeout' must be a non-negative number

The error does not appear immediately after startup, but after some time.

When connecting to ES Cloud cluster, the client produces this error:

ValueError: root_certificates is required for secure connection

This requirement must not be there as root certificate is only required when connecting to a cluster or instance, which is using an untrusted certificate. Because ES Cloud is using certificates signed by a public CA, users don't even have access to the root certificate (no one does).

I am not sure if it's related to ES Cloud only (it shouldn't) but I have a report of an exception:

Traceback (most recent call last):
File "/home/ubuntu/.local/lib/python3.10/site-packages/esdbclient/client.py", line 204, in _connect_to_preferred_node
answers = dns.resolver.resolve(cluster_fqdn, "A")
File "/home/ubuntu/.local/lib/python3.10/site-packages/dns/resolver.py", line 1565, in resolve
return get_default_resolver().resolve(
File "/home/ubuntu/.local/lib/python3.10/site-packages/dns/resolver.py", line 1307, in resolve
(request, answer) = resolution.next_request()
File "/home/ubuntu/.local/lib/python3.10/site-packages/dns/resolver.py", line 749, in next_request
raise NXDOMAIN(qnames=self.qnames_to_try, responses=self.nxdomain_responses)
dns.resolver.NXDOMAIN: The DNS query name does not exist: clt07f5o0aesbufe1n6g.mesdb.eventstore.cloud:2113.

I checked the line 204 and it calls answers = dns.resolver.resolve(cluster_fqdn, "A"). From the error message I understand that cluster_fqdn also contains the port, which is apparently wrong.

The code that calls the client looks totally fine, so I am not sure what went wrong:

client = EventStoreDBClient(
    uri="esdb+discover://admin:something@<cloud-cluster-id>.mesdb.eventstore.cloud:2113"
)

We tried to use esdb://<cloud-cluster-id>-0.mesdb.eventstore.cloud:2113 as I had a guess that esdb+discover might be an issue, but it crashed with the same error. It tried to resolve <cloud-cluster-id>-0.mesdb.eventstore.cloud:2113, again, including the port in DNS query.

I know in event store you can subscribe to $et-EventType stream to get all those events.

However, when I try to do that with esdbclient:

stream_name = "$et-EventType"
    events = client.get_stream(
        stream_name=stream_name
    )
    event_types = []
    for event in events:
        print(event)
        print(event.data)

I only get the strange binary string with a 0@uuid in it. I believe it is a link to the event, but I'm not sure how to get the actual event body:

RecordedEvent(type='$>', data=b'0@90a1af0f-f218-443e-8805-f2a419d78833', 
metadata=b'{"$v":"4:-1:1:4","$c":21867,"$p":21867,"$causedBy":"8e8c01d8-7819-4b7e-a891-d88a5cf6a9eb"}', 
content_type='application/octet-stream', id=UUID('ca25115c-fd64-4216-8361-132101384c00'), 
stream_name='$et-EventType', stream_position=0, commit_position=22961, retry_count=None)

If I created event with:
NewEvent(type=MyEvent, data=b"") and append it to stream then this event is not coming to Persistent subscription.