Upgrade to PHP 7.4 (property typehints) about event-store-client HOT 17 CLOSED

Thanks for all your feedback. A few notes: protobuf is avaible not only via pecl, but also via packagist as a library (in pure PHP). That said, it should be possible to run this on PHP 7.4 right now and when the extension gets updated, than that will work as well.

As I don't want to create a new major version every year (working on this one for 1.5 years! already), it's easier for me to do this once now and require PHP 7.4. At least I would be happy with it. However I wanted to ask, maybe someone has a real blocker or can't migrate as quick. Nowadays most people use docker containers anyway.

from event-store-client.

Longterm: yes
Shortterm: Protobuf isn't even compatible with 7.4 yet. Also PHP x.y.0 versions are known to be quite buggy so better wait a while.

from event-store-client.

Well I am not using this package anymore, because the personal PoC project I was using it in turned out to be not so viable (at least in PHP, for various reasons that had nothing to do with this package). So for me it doesn't matter.

But for a general point of view, and do with this information whatever you want, is that RHEL/CentOS is only just now getting support for PHP 7.3. It will probably be another year before 7.4 is added. At my work we are bound to Red Hat due to customer requirements, and for compliancy reasons we can only install official packages (so no Remi and no compiling from source). So there we cannot even use packages requiring 7.3 (in December we should see 7.3 coming to RHEL 7 and then we can finally upgrade).

from event-store-client.

You can contact me if you have any more questions, happy to provide additional insights :-)

from event-store-client.

@codeliner To consume an EventStore projection you have two options:

1. Start polling via an HTTP API
2. Start listening on a TCP connection and wait for EventStore to push to you whenever something arrives.

Option 1 isn't viable if you want your consumer to consume events in (near) real-time. You'll just be hammering EventStore with HTTP requests which is very slow and very inefficient. It just doesn't scale well.

Option 2 because of how the EventStore TCP connection works is only possible in an asynchronous way, which is not natively supported by PHP. Now the guys from AMP created an amazing package to allow for asynchronous programming in PHP, but it requires such a different approach to writing code that it limits you a lot in the external packages you can use. For example if you want to have a MongoDB connection, there is no AMP package for that. Or if in some cases you don't want to use EventStore as a message broker and want a separate AMQP connection, there also is no package for that. Using a package that's not written for AMP (or ReactPHP) also isn't an option because that'll be written in a blocking way, giving problem to the asynchronous setup.

I ended up writing a layer that would consume an event, and then spin off a new PHP process using amphp/process that would run in an old-fashioned synchronous way. Then in the sub-process you can use any package from Packagist you want, without having to worry about breaking or creating a blocker in the asynchronous process. For this to work I also wrote a communication layer that allowed the main asynchronous process to communicate with the child process by sending and receiving DTOs over standard input/output. It worked really nicely and from a design perspective (everything needed to be Onion/Hexagonal) I was quite happy with the result. However a solution like that is probably going to end up being problematic to maintain in a bigger project. Also spinning off sub-processes doesn't tend to scale well when using PHP in terms of CPU and memory overhead that it requires per process. So The PoC itself was great, but it was clear there were going to be too many maintainability and scaling issues to make it viable enough to move ahead with it, at least in PHP. That's why I abandoned that project. This package itself was really great, but keeping everything AMP style proved too much for the size of the eventual project that I had in mind.

from event-store-client.

@SunMar

Or if in some cases you don't want to use EventStore as a message broker and want a separate AMQP connection, there also is no package for that.

AMP package: https://github.com/phpinnacle/ridge
ReactPHP package: https://github.com/jakubkulhan/bunny
(Bunny seems to work fine with AMP too.)

Using a package that's not written for AMP (or ReactPHP) also isn't an option because that'll be written in a blocking way, giving problem to the asynchronous setup.

That's not necessarily an issue for single-purpose commands such as consuming a projection.

I ended up writing a layer that would consume an event, and then spin off a new PHP process using amphp/process that would run in an old-fashioned synchronous way.

See https://amphp.org/parallel/workers

from event-store-client.

ping @codeliner @SunMar @pkruithof @enumag @sandrokeil @burzum @bogdangrigore @SlayerBirden @gitomato

from event-store-client.

I guess folks who run PHP 7.3 already will upgrade to 7.4 quickly. If protobuf does not compile yet, this is a blocker, right? So we have no choice. Otherwise I'd set 7.4 as minimum to be able to use typed properties.

from event-store-client.

I'd wait for the first minor release before requiring it, or at least some patch releases.

Well I am not using this package anymore, because the personal PoC project I was using it in turned out to be not so viable (at least in PHP, for various reasons that had nothing to do with this package).

We're still using it (in production) but are going to migrate away from Eventstore completely as we found it to be unreliable. We had multiple occasions where events were not properly distributed across projections, the correlation projection faulting multiple times, and lots of smaller issues, some of which we actually reported bugs to the Eventstore team (some are fixed, others not).

Bottom line: we cannot get it to operate in a reliable, high-available cluster. And when for instance memory issues arise, Eventstore can actually corrupt your streams, which should never ever happen in a datastore.

from event-store-client.

Wow, very interesting insights. What is your alternative to EventStore @pkruithof ?

@SunMar can you share why it was not viable? You already said it has nothing to do with the package. But anyway, if possible I'd love to know the reason. Might be a very important information for us.

from event-store-client.

What is your alternative to EventStore @pkruithof ?

We're looking at cloud-hosted solutions because we don't want to host or manage these kinds of datastores anymore. Right now we're experimenting with Google Firestore, using a very basic implementation based on the blueprint that I think you wrote, correct?

So far the performance is far from comparable: Eventstore is much faster to write to or read from, especially when you run it next to your application. But it could not cope with my benchmarks: I kept getting concurrency and availability related errors. Firestore just ate everything we threw at it without any hitch.

Faster is better of course, but the command side of your application usually does not have to be as fast that tens of milliseconds matter that much. So it's a trade-off that's worth it imo, especially when data integrity is not ensured for Eventstore.

from event-store-client.

By the way, I'm sure that you can run Eventstore reliably in certain setups. But I think that's mostly on a very high specced vps/server cluster in a rack next to your application. You simply have to spend a lot of money on it.

The thing is that if your datastore cannot handle data integrity unless it is hosted in a specific kind of way, it's not ok to use it for critical parts of your application.

from event-store-client.

We are very happy with our MongoDB-Cluster running on AWS EC2 especially since MongoDB 4.x (transaction support was added), but the cluster is managed by the team.

For our new SaaS product InspectIO we start with proop/event-store v7 Postgres version and try AWS RDS Aurora, but want to switch to v8 with EventStore later. Hence, your feedback is very interesting for me. @prolic was very happy with EventStore in production. But I read similar concerns about EventStore before. You're not the only one reporting issues.

from event-store-client.

@enumag Those points don't address the core of my concerns. But my initial reply was because @codeliner asked, I don't want to take over this issue with more off-topic stuff. If you're interested I'm more than happy to elaborate / deep dive into this elsewhere :).

from event-store-client.

thx @pkruithof and @SunMar

from event-store-client.

Note: Protobuf extension is compatible with 7.4 now. Also PHP is now at 7.4.2 so I think it's time to do this change. I could attempt to do it using Rector but I'd prefer this PR to be merged first.

from event-store-client.

done.

from event-store-client.