iamgroute / nptpy Goto Github PK
View Code? Open in Web Editor NEWLicense: GNU General Public License v3.0
License: GNU General Public License v3.0
NPTpy/NPTpy/Portal/DataEndpoint.py
Lines 26 to 32 in 68f068f
Setting to blocking mode to send was temporary and has to be remove.
If the device (to which the data is being sent) stops receiving, send
will block indefinitely.
To remove the blocking send
we need per-channel flow control.
According to https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json, Python 3.6 requires Ubuntu 20.04 not 22.04 which is currently ubuntu-latest
.
Maybe specify matrix for distros?
If an exception occurs (at least at the stage of authenticating a Link
portal-side), and is not handled before it reaches portal.main.py
, main()
short loops after recovery (after time.sleep(10)
).
Perhaps there is a reference cycle that prevents gc from closing some sockets ?
TCP allows for one endpoint to send FIN and terminate the communication in one direction, while still allowing for data to be sent the other direction. Some applications use this to signal EOF.
We could implement this by calling shutdown(read=False, write=True)
when the control endpoint receives a channel deletion request.
However, we need to avoid causing a short-loop on the portal next to the device that sends the first FIN.
When a request is pending and link drops:
1604085298.3 [00000073] DataEndpoint: Finishing ()
1604085298.3 [00000000] Loop: Enqueue (8,)
1604085298.3 [00000000] Loop: Running (8, 'requestCloseChannel ControlEndpoint:129')
1604085298.3 [00000000] Loop: NewFuture ()
1604085298.3 [00000000] Loop: Paused (8, 'requestCloseChannel ControlEndpoint:136')
.
1604085298.4 [00000066] Connector: Error (ConnectionResetError(104, 'Connection reset by peer'),)
1604085298.403 [Portal.Link ] Connection lost, reason: Closed by other end
1604085298.404 [Portal.Link ] Reconnecting
1604085298.4 [00000000] Loop: Enqueue (10,)
1604085298.4 [00000000] Loop: Running (10, 'reconnect Link:163')
1604085298.4 [0000002A] Link: Disconnect ()
1604085298.4 [00000066] Connector: [Log deleted]
1604085298.4 [00000000] Loop: Enqueue (8,)
1604085298.4 [00000073] DataEndpoint: [Log deleted]
1604085298.4 [00000076] Connector: [Log deleted]
1604085298.4 [0000002A] Link: Connect (None,)
1604085298.4 [00000000] Loop: NewFuture ()
1604085298.4 [00000000] Loop: Paused (10, 'reconnect Link:165, __call__ Event:26, _connect Link:62, _connect_p1 Link:74, requestRelay Portal:241')
1604085298.4 [00000000] Loop: Running (8, 'requestCloseChannel ControlEndpoint:136')
1604085298.4 [00000000] Loop: RunError (CancelledError(),)
.
By separating the 'client' and 'permanent' conditions, it's possible to have links that are configured and maintained (re-established after disconnect) on both ends.
Exception ignored in: <function Promise.__del__ at 0x7616d738>
Traceback (most recent call last):
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 19, in __del__
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 57, in fire
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 55, in fire
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 119, in <lambda>
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 126, in _resolve
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 33, in then
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 29, in attach
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 55, in fire
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 126, in <lambda>
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 132, in _cont
File "/home/pi/git/NPTpy/NPTpy/Common/Async.py", line 141, in run
RuntimeError: cannot reuse already awaited coroutine
After some stress testing:
...................
1604087477.8 [00000031] Listener: NewConnection ()
1604087477.8 [00000000] Loop: Enqueue (22,)
1604087477.8 [00000000] Loop: Running (22, 'main Listener:49')
1604087477.8 [000000FB] Endpoint: [Log created]
1604087477.8 [000000FB] Endpoint: Inited (7, -1)
1604087477.8 [00000000] Loop: NewFuture ()
1604087477.8 [00000000] Loop: Paused (22, 'main Listener:50, requestChannel Link:221, requestChannel Channels:100, requestNewChannel ControlEndpoint:67')
.
1604087477.9 [000000EA] Connector: Error (ConnectionResetError(104, 'Connection reset by peer'),)
1604087477.942 [Portal.Link ] Connection lost, reason: Closed by other end
1604087477.943 [Portal.Link ] Reconnecting
1604087477.9 [00000000] Loop: Enqueue (28,)
1604087477.9 [00000000] Loop: Running (28, 'reconnect Link:165')
1604087477.9 [0000002A] Link: Disconnect ()
1604087477.9 [00000000] Loop: RunError (AttributeError("'Endpoint' object has no attribute 'readable'"),)
..
1604087483.0 [00000000] Loop: Enqueue (23,)
1604087483.0 [00000000] Loop: Running (23, 'requestCloseChannel ControlEndpoint:136')
1604087483.0 [00000000] Loop: RunError (CancelledError(),)
.
1604087485.7 [00000000] Loop: Enqueue (26,)
1604087485.7 [00000000] Loop: Running (26, 'requestKA Portal:244')
1604087485.7 [00000000] Loop: Finished (26, None)
1604087485.8 [00000000] Loop: Enqueue (1,)
1604087485.8 [00000000] Loop: Running (1, 'rtask Portal:167, recvPacketAsync AsyncConnectorPacketized:30, bufferAsync AsyncConnectorPacketized:19, tryRecvAsync AsyncConnector:93, recvAsync AsyncConnector:84, wait AsyncConnector:16')
1604087485.8 [00000000] Loop: NewFuture ()
1604087485.8 [00000000] Loop: Paused (1, 'rtask Portal:167, recvPacketAsync AsyncConnectorPacketized:30, bufferAsync AsyncConnectorPacketized:19, tryRecvAsync AsyncConnector:93, recvAsync AsyncConnector:84, wait AsyncConnector:16')
..
1604087490.8 [00000000] Loop: Enqueue (13,)
1604087490.8 [00000000] Loop: Running (13, 'requestCloseChannel ControlEndpoint:136')
1604087490.8 [00000000] Loop: RunError (CancelledError(),)
.1604087495.781 [root] 'NoneType' object has no attribute 'socket'
Traceback (most recent call last):
File "portal.main.py", line 38, in <module>
p.main()
File "/home/pi/git/NPTpy/NPTpy/Portal/Portal.py", line 56, in main
Globals.runReminders()
File "/home/pi/git/NPTpy/NPTpy/Globals.py", line 12, in runReminders
kaReminderTX.run()
File "/home/pi/git/NPTpy/NPTpy/Common/TimedReminder.py", line 13, in run
Reminder.run(self)
File "/home/pi/git/NPTpy/NPTpy/Common/Reminder.py", line 24, in run
delegate.run()
File "/home/pi/git/NPTpy/NPTpy/Common/Reminder.py", line 56, in run
if f is not None: f(owner)
File "/home/pi/git/NPTpy/NPTpy/Portal/Link.py", line 195, in handleRemindTX
self.channels.sendKA()
File "/home/pi/git/NPTpy/NPTpy/Portal/Channels.py", line 55, in sendKA
self.epControl.sendKA()
File "/home/pi/git/NPTpy/NPTpy/Portal/ControlEndpoint.py", line 54, in sendKA
self.send(b'.', True)
File "/home/pi/git/NPTpy/NPTpy/Portal/ControlEndpoint.py", line 23, in send
self.parent.send(self.formMessage(data), untracked)
File "/home/pi/git/NPTpy/NPTpy/Portal/Channels.py", line 52, in send
self.myLink.send(data, untracked)
File "/home/pi/git/NPTpy/NPTpy/Portal/Link.py", line 269, in send
self.conRT.socket.settimeout(5)
AttributeError: 'NoneType' object has no attribute 'socket'
Caused by e89c4b7 ?
Trace:
... (many keepalives)
1568793504.496 [Portal.Link ] Sent 5 Bytes
1568793504.497 [Portal.Link ] content: 0100000000
1568793504.498 [Portal.Link ] Sent 5 Bytes
1568793504.499 [Portal.Link ] content: 0100000000
1568793511.769 [Portal.Link ] Received 5 Bytes
1568793511.770 [Portal.Link ] content: 0100000000
1568793513.801 [Portal.Link ] Received 5 Bytes
1568793513.801 [Portal.Link ] content: 0100000000
1568793514.887 [Portal.Link ] Sent 5 Bytes
1568793514.888 [Portal.Link ] content: 0100000000
1568793514.889 [Portal.Link ] Sent 5 Bytes
1568793514.890 [Portal.Link ] content: 0100000000
1568793525.391 [Portal.Link ] Sent 5 Bytes
1568793525.392 [Portal.Link ] content: 0100000000
1568793525.393 [Portal.Link ] Sent 5 Bytes
1568793525.394 [Portal.Link ] content: 0100000000
1568793535.892 [Portal.Link ] Sent 5 Bytes
1568793535.893 [Portal.Link ] content: 0100000000
1568793535.895 [Portal.Link ] Sent 5 Bytes
1568793535.895 [Portal.Link ] content: 0100000000
1568793546.356 [Portal.Link ] Sent 5 Bytes
1568793546.357 [Portal.Link ] content: 0100000000
1568793546.358 [Portal.Link ] Sent 5 Bytes
1568793546.359 [Portal.Link ] content: 0100000000
1568793556.831 [Portal.Link ] Sent 5 Bytes
1568793556.832 [Portal.Link ] content: 0100000000
1568793556.833 [Portal.Link ] Sent 5 Bytes
1568793556.834 [Portal.Link ] content: 0100000000
1568793561.024 [Portal.Link ] RX keepalive timeout
1568793561.028 [Portal.Link ] reconnecting
1568793561.028 [Portal.Link ] Stopping
1568793561.030 [Portal.Link ] Stopped
1568793561.030 [Portal.Link ] Stopping
1568793561.031 [Portal.Link ] Stopped
1568793561.035 [Portal.Portal ] Sent 64 Bytes
1568793561.036 [Portal.Portal ] content: 01000000765432103030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030
1568793561.036 [Portal.Link ] RX keepalive timeout
1568793561.039 [Portal.Link ] disconnecting
1568793561.040 [Portal.Link ] Stopping
1568793561.041 [Portal.Link ] Stopped
1568793561.042 [Portal.Link ] Stopping
1568793561.042 [Portal.Link ] Stopped
1568793562.391 [Portal.Portal ] Received 39 Bytes
1568793562.391 [Portal.Portal ] content: 76302E316F70696F206C81D767869F38BB0172302E736572766572732E6E6574706F72742E696F
1568793562.396 [Portal.Link ] Starting
1568793562.397 [Portal.Link ] Started on [0.0.0.0]:40361
1568793562.397 [Portal.Link ] Connecting to [r0.servers.netport.io]:443
1568793562.456 [Portal.Link ] connected
1568793562.457 [Portal.Link ] Sent 64 Bytes
1568793562.458 [Portal.Link ] content: 206C81D767869F383030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030303030
1568793562.501 [root] argument must be an int, or have a fileno() method.
Traceback (most recent call last):
File "portal.main.py", line 31, in <module>
p.main()
File "/NPTpy/NPTpy/Portal/Portal.py", line 69, in main
readables, _, _ = select.select(rlist, [], [], 1) # Temporary
TypeError: argument must be an int, or have a fileno() method.
In some cases it takes way too much time (> 2 minutes) to detect a loss of communication.
It is also possible for the detection to be missed entirely if other data is sent (but not received) within this period, as the keep-alive counters are reset.
1566313813.999 [root] do_handshake_on_connect should not be specified for non-blocking sockets
Traceback (most recent call last):
File "NPTpy/NPTpy/Portal/Portal.py", line 53, in main
s.task()
File "NPTpy/NPTpy/Portal/Link.py", line 159, in task
elif self.state == self.States.WaitReady: self.taskReady()
File "NPTpy/NPTpy/Portal/Link.py", line 170, in taskReady
if data == b'Ready !\n': self.secureForward()
File "NPTpy/NPTpy/Portal/Link.py", line 84, in secureForward
self.conRT.secureServer(certFilename='portal.cer', keyFilename='portal.key')
File "NPTpy/NPTpy/Common/Connector.py", line 176, in secureServer
self.socket = self.sslContext.wrap_socket(self.socket, server_side=True)
File "/usr/lib/python3.6/ssl.py", line 407, in wrap_socket
_context=self, _session=session)
File "/usr/lib/python3.6/ssl.py", line 816, in __init__
raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
ValueError: do_handshake_on_connect should not be specified for non-blocking sockets
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.