OPCUA Subscription limit about opcua-asyncio HOT 12 CLOSED

depends entirely on your CPU / Ram

edit: maybe on the Python Version

from opcua-asyncio.

I have 4 vcpu / 16 GB machine and Python version 3.8. This configuration works great for 3000 tags. At 6000 tags, This is creating lag

from opcua-asyncio.

Thanks for the fast reply. This is with a single process setup and using all tags in single subscription

from opcua-asyncio.

Subscription settings:
PublishInterval?

MonitoredItem settings:
SamplingRate?
QueueSize?
Filter?

All these settings can influence the Performance!

Hint:
">=Python 3.10" as some Performance improvements
Changing the Interpreter is the fastest way to increase Performance ;)

from opcua-asyncio.

PublishInterval = 1 second
sampling_interval = 500 millisecond
queuesize = 50
no filter as such all updates are received
I agree with the interpreter part. But just thinking if it is possible to scale by settings or by adding multiprocessing.

from opcua-asyncio.

scaling is working by using multiple clientconnections/instances sharing the load of nodes to monitor! opcua-asyncio is using an EventLoop which makes multiprocessing... how to say politely ^^ Not a path you want to go...

guess your using Linux which has another large influence on the EventLoop performance!?

from opcua-asyncio.

I have another process that pushes to remote kafka all received datapoints. Just to note if i use 8 vcpu, it is again working like a charm with kafka receiving at below 3 second latency at 6000 datapoints

from opcua-asyncio.

If i were to split the load in 2 two processes into 3000 tags each, it is consuming too much cpu causing delays in other processing. Is that supposed to be the case?

from opcua-asyncio.

actually it should enable the OS to run the two Python-Processes (a Client-Instance each) on different CPU Cores (tbh not sure how vCores work in that regard)

from opcua-asyncio.

Good point. I too was thinking along the same lines and expected performance improvement because of 4 vcores. Unexpected cpu rise was the case though. Will debug it more on why that was the case in vcpus. Is there any testing metrics i can go with to see whether my code is doing fine for single subscription with 6000 tags?

FreeOpcUa/python-opcua#701 (comment)
Is such testing done by any chance?

from opcua-asyncio.

usually the server exposes their limits how many monitoreditems it supports per subscription...

https://reference.opcfoundation.org/Core/Part5/v104/docs/6.3.11
its part of the server object -> ServerCapabilities -> OperationalLimits

but just a numbers game -> if you have like 5000 msg/s you have 5ms time per message this is a lot for python (plus on top of that you have network delay to kafka) as long as you do just pushing towards kafka i would use a async generator (tends to have the best performance) to fill the gap between us-client and kafka

from opcua-asyncio.

Got it thanks

from opcua-asyncio.