An Erlang implementation of libp2p swarms
Home Page: https://helium.github.io/erlang-libp2p
License: Apache License 2.0
I'd be nice to be able to have streams time out and close after some specified period of inactivity
What is the plan to add custom transports? I'm thinking to add support for websockets to the lib but I'm wondering if such support should be internal or via an extrenal app?
{timeout,{gen_server,call,[<0.578.0>,{dial,"/ip4/127.0.0.1/tcp/49567","sync/1.0.0/3266/c700f5995b756ef608841554bd4ea09aad0fa02375cf30f4dc2d4102a93dab5b"}]}}
I'm wondering if you have a list of todo thing and things that are expected to work right now? I would like to try to use it in semi production with just the tcp protocol support for now. Let me know :)
I'd be nice to test interop with Go and JS implementations and possibly even make an integration test we can run to confirm things are still compatible.
Chain syncing is painfully slow for a painfully long time. Can we just force it to sync with fast connecting local nodes?
I have the following scenario in mind :
Is there a way to know if a channel is open to A on R and use it? Is this the right way to do it?
Like bitcoin, limit the size of the peerbook to a maximum number (default 1000?). If new peer entries come in, cycle out old ones.
The relcast group concept seems badly overloaded by trying to make it general enough to support relcast messaging.
While we would still like for relcast to use libp2p as at least an optional transport, it would be best to move the group management and reconnection logic into a supervisor in the relcast application.
1> S1 = libp2p_swarm:start(0).
<0.333.0>
15:17:18.982 [info] Started Listener on "/ip4/127.0.0.1/tcp/53110"
2> S2 = libp2p_swarm:start(0).
15:17:18.983 [info] Started Listener on "/ip4/127.0.0.1/tcp/53111"
<0.351.0>
3> [S2Addr] = libp2p_swarm:listen_addrs(S2).
["/ip4/127.0.0.1/tcp/53111"]
4> {ok, Session1} = libp2p_swarm:connect(S1, S2Addr).
15:17:19.927 [info] Connecting to "/ip4/127.0.0.1/tcp/53111"
15:17:19.929 [info] Starting client session with "/ip4/127.0.0.1/tcp/53111"
15:17:19.931 [info] Starting handshake with client "/ip4/127.0.0.1/tcp/53112"
15:17:19.932 [info] Negotiating handler for "/ip4/127.0.0.1/tcp/53111" using ["yamux/1.0.0"]
15:17:19.932 [info] Negotiated server handler for "/ip4/127.0.0.1/tcp/53112": "yamux/1.0.0"
{ok,<0.371.0>}
5> 15:17:24.935 [warning] Unhandled message: timeout
15:17:24.936 [warning] Unhandled message: timeout
15:17:24.936 [error] gen_server <0.371.0> terminated with reason: unknown
15:17:24.936 [error] gen_server <0.370.0> terminated with reason: unknown
15:17:24.967 [error] CRASH REPORT Process <0.371.0> with 0 neighbours exited with reason: unknown in gen_server:handle_common_reply/8 line 726
15:17:24.967 [error] gen_server <0.370.0> terminated with reason: unknown in gen_server:handle_common_reply/8 line 726
15:17:24.967 [error] CRASH REPORT Process <0.370.0> with 0 neighbours exited with reason: unknown in gen_server:handle_common_reply/8 line 726
15:17:24.967 [error] gen_server <0.372.0> terminated with reason: unknown
15:17:24.968 [error] CRASH REPORT Process <0.372.0> with 0 neighbours exited with reason: unknown in gen_server:decode_msg/9 line 410
15:17:24.968 [error] Supervisor {<0.335.0>,libp2p_swarm_session_sup} had child "/ip4/127.0.0.1/tcp/53111" started with {libp2p_yamux_session,start_client,undefined} at <0.371.0> exit with reason unknown in context child_terminated
15:17:24.968 [error] gen_server <0.373.0> terminated with reason: unknown
15:17:24.968 [error] CRASH REPORT Process <0.373.0> with 0 neighbours exited with reason: unknown in gen_server:decode_msg/9 line 410
15:17:25.030 [error] Ranch listener "/ip4/127.0.0.1/tcp/53111" terminated with reason: unknown
Seed nodes, routers, etc, can often not have their peer books gossipped because the peer book entry is too large. Do something about this so the peer book entries are under the size limit so their peer entries can be gossipped.
Ranch allows a maximum number of acceptors to be set, which limits the number of sessions.
Make this configurable for the swarm. #9 deals with this for limiting the number of streams in a session
noticed an ever increasing number of peerbook entries with no listen address specified
I've inserted the following debug code in to libp2p_peerbook:put to monitor arp results / gossip for updates to gateways which were previously dialable but are not now:
case NewPeerId /= ThisPeerId
andalso libp2p_peer:is_dialable(ExistingPeer)
andalso not libp2p_peer:is_dialable(NewPeer) of
true ->
lager:info("old peer dialable, new peer not ~p", [libp2p_crypto:pubkey_bin_to_p2p(NewPeerId)]);
_ ->
ok
end,
and seeing a huge number with no listen address:
2021-11-27 18:34:18.918 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/1121r64SZuULMcUDpnqJDTV1yp4oKzk1BSxYjuFd4vUfJxLzZrTd"
2021-11-27 18:34:18.919 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/118QsxBeZDKu1HPc1kUzwVNP4oAEnZaCPM9TooW3gn8zz25PJfe"
2021-11-27 18:34:18.930 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11kdmFytQbJonGfN3XiAFSWx9zFYRxXG8oDrswnStih6PbcbyNn"
2021-11-27 18:34:18.937 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11BKKMZa9PpY6GkzLeruRkE67r32v9obHkrKwuX2rgcDdh6HgS1"
2021-11-27 18:34:18.941 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/1128uozMn22zm6rhFpzt4ixbdckzFAYwPT7uiiSrv6EWW77oCSZF"
2021-11-27 18:34:18.941 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/119NCQkWvUJvV3uTkVKn7PhNccRJwDqUe8tHzWsbubvUqH83Swh"
2021-11-27 18:34:18.942 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112uQ3DBo5tuCCpfF7JgiH2sMDcDyCNAnJbbTKak3F88Eh9W9Lp9"
2021-11-27 18:34:18.943 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11rF1XdTVEdu81EpziKno7WBfe5K8yaKFgsuvi5Xj5cjYcdyDKT"
2021-11-27 18:34:18.957 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/116p11JMjzZSBF8e4Wsej5TSkWzoUWbz4rC8SoGm8MMm9J5XvXA"
2021-11-27 18:34:18.958 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11KgMKRpQAzhi7gJ4ybWZJis3zZbn1mqLt3fFJYZ8Hg8V3Gkmv4"
2021-11-27 18:34:18.958 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11qMPu2NNC3ddvvuAG9JvQbrQnctSWygdHd69hT3LL9zaxjwPcU"
2021-11-27 18:34:18.960 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/116f3mCqkU4oohkZXaPMqVzBqo2cnfiiA9HzEPkCaP1qSKd2VtL"
2021-11-27 18:34:18.969 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112kLY6pzoSSGmLpNSSMR9PaFaPkLvj8dR15xgWVqzFeB8VqxZ6M"
2021-11-27 18:34:18.973 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11pj9SLws6dnqDw3Rt6tmQTCBFhewfSkzmDBJ7nCnMuUDRkkStg"
2021-11-27 18:34:18.980 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/1124pFghWynjqqz6H9YQD4LtwU65UCUDpiSq5ZkYapD7cUDn37EP"
2021-11-27 18:34:18.984 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11pHooW9ftksFxLEyYTYRATKecyP91WHCydCE758j3BKdXCpYQD"
2021-11-27 18:34:18.993 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112TWaKbTPzeq39SBtDi5soMfikBLcFTvCPUX4U86z59ncoxJxX7"
2021-11-27 18:34:18.995 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11WaWVa5znPRtesK9F5Maq31iDVivFyKRDFajsn75Dk8FWVqFcR"
2021-11-27 18:34:18.995 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11JbRyC7Rf2T8y81jnkTreHt5YL7w7RzG9wwRbJ4NoRSnbLNQbU"
2021-11-27 18:34:19.001 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11eJCM2GiCFvnaWghfV2Tg86XACKiDdT1Gp3YLptT6wwedGxJgV"
2021-11-27 18:34:19.013 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112BVsAT7E1MPZCqiWUwMqESMyFPV2JkoaSuU8K2LJgfaemoU13Y"
2021-11-27 18:34:19.013 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11CQ4hU6vSRNu4cT5b6gKyMgb9ThuMQbgzsniWwemxbAHHGMzsE"
2021-11-27 18:34:19.015 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112HqZNAz92X3EEU9EnbJgd1gqRQx1eD5JztNwuYDVbr3o1gEySM"
2021-11-27 18:34:19.017 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112Kpr39queLAVnE6Hcuxgrmp8tfCuRqRq38twXC9p9q9cm493UD"
2021-11-27 18:34:19.018 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112LRCzntrpjAsMgtcUV5K8W37WPx9Dhwiv1DW9nsFvyoM1Yr78F"
2021-11-27 18:34:19.020 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112mJwJmR3knaQ8yA52jzi75r3mx3bWSkuo86rFNVQLxKHiyPUS3"
2021-11-27 18:34:19.021 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112sEHzqqHHtmX4NhCbcxDFzeHranoknR8iwKdy7gkF3YsLVPEAw"
2021-11-27 18:34:19.023 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112HSZ7G3UKnHARzUJ3urGEF6RquC2nU65eQCAMmay6ZTuMcEfZT"
2021-11-27 18:34:19.023 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112QXUeLe37bv6HGyRgtFzLr7eh4Nkxu1YX5HuLFw4Li8yS1utdq"
2021-11-27 18:34:19.027 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11jGiJyJPu3fsT3veVZYxyc7XrGwtfigXfU1TUSBFicycDqtBc"
2021-11-27 18:34:19.027 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11M2dGD3bxBc435wYjF1AeqRBkPkCtzCgopfj44Qh17M3VbFtZd"
2021-11-27 18:34:19.028 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112W5q1VefJWdbu7NLcVv5XjSqk7z7vj58zdtmuZrKwh1sY18J47"
2021-11-27 18:34:19.033 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11KJy6x5kskxULJHzBx2bvdtaqbNc5LMio1bDB9jDhKQKX71f1u"
2021-11-27 18:34:19.034 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112ZjWqv7WNjxdro6jeaoT5gGvEyu9escfb9QYD7FM98197vjQUi"
2021-11-27 18:34:19.037 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112injVkSRKQu9xSipXgGkQ3jbLydC9bbwWSxXnHEtnutXq5Du6D"
2021-11-27 18:34:19.051 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11bEr3f2QCrkjrJfGFTMycMKjiwinurFC9SptBVeab97FQanc62"
2021-11-27 18:34:19.056 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112sfgtnz7oHSSxq7Uked2Q5dSy32jtXdnSx3RHGrKQfWfw4TaAf"
2021-11-27 18:34:19.086 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112TiFEb6uEtCP4o5Vrwb9yXJnr3Sn1wQ1XdYu4yEEFHpYSUL2mX"
2021-11-27 18:34:19.087 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112QuZLRFDzX57Z5RJxsPEqznyJC5PY62ENCv1wvZCB99CYrYc5w"
2021-11-27 18:34:19.093 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11NeTetAdSA6wXJWdF8dgzo1CgMJ3eCJr6rn8Mrh81VhiDoDWr8"
2021-11-27 18:34:19.093 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112nvMsLWADEziaMEYXsKhQ4shN89n54cLzMoKRR8WZTGLZ9HPsn"
2021-11-27 18:34:21.184 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112ovzjSKvJC2fgwC5hHveAr9nLUfVmN2UZJCg3S1L5jmm7aqcMY"
2021-11-27 18:34:21.190 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112YPcT8VbAd1RLzjjA7ZxVWjudYNv1W1NJtJyC74TvLcxuSQN4H"
2021-11-27 18:34:21.202 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11DP2WAmj1YvBqgCopu68UihwxfoLVKuQkCUSqyxsmHdYF7n3p4"
2021-11-27 18:34:21.242 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11vAcU79e9gN1V3ywUkXQ9QL2bDPxegPLmC7K19rn5yrSMbX8sg"
2021-11-27 18:34:21.280 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11ZpDt7E97aACzzCML1jYdvD8PXs6BL6N4xo3vgnwRYveC6UFKp"
2021-11-27 18:34:21.281 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11FtTiyDRbvVkeUuG7bU5GPzKPwwT8ibJ7mcVcL3GQiMT7MR6Bu"
2021-11-27 18:34:21.285 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11NZKNS5cse6osHrHw8pxdyjCvKU8rAzG6s7Zy9my2gJKL3gyJb"
2021-11-27 18:34:21.291 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11vZxuqmJvDHtJFzsDGHiHEmupaGUXZWgJtXXJgAHoZdZqfCTwJ"
2021-11-27 18:34:21.293 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/113XfiXQ3AEEc9PH978SifS6YpxxzUUR7DKzw2qp9TuFBYu87dV"
2021-11-27 18:34:21.299 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/117DToxyE2JYV6WdxDJQUM3ftivyP3wK73AzRJGaz8wXtkxbk67"
2021-11-27 18:34:21.307 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11iG18HXnvUejygHmYD4uxtTx5ZqFDk3kruUXXxWUGbn7y8aqMG"
2021-11-27 18:34:21.314 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11jphBNVhMt6RyPiBytFjXGTQ8DgYKasaSbZxui4iPoDoHi39Rh"
2021-11-27 18:34:21.323 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/1127pHxysgctLdfx6NvBeLUVorQiHmBpGLSwfkwuvpATkcyZxStf"
2021-11-27 18:34:21.338 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11nEGj5tRrYfnGDgQ1zMzqAS7mJ8qkFYSzDK46ZgtBGRSNVKKVJ"
2021-11-27 18:34:21.339 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11WaWVa5znPRtesK9F5Maq31iDVivFyKRDFajsn75Dk8FWVqFcR"
2021-11-27 18:34:21.355 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112fwsUHux2uFmhJbXBGxZ9W37KZbNUUAzKTXmxwnJijcH7G5RWb"
2021-11-27 18:34:21.356 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112nnF1Cz6Kf5NEL5U7FuMT4Uv5h2QAW5Mrp6HpgEqMmsG6fr1Qz"
2021-11-27 18:34:21.359 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112N1VVjSpy4YAx3ZvZyxBjG3fNzwJ5Jt5ecp9utuyHxmURgcwfY"
2021-11-27 18:34:21.360 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/11iGz7XLS3PSwMfHFqjqjgZB63CXCePj8PfKfhayU3uf7oeJMSH"
2021-11-27 18:34:21.367 [info] <0.1284.0>@libp2p_peerbook:put:121 old peer dialable, new peer not "/p2p/112d96iTNg62u1EDMw2F3A7xWmKYJuxzAitYxeXg8bZLCDaJSvnc"
because libp2p_peerbook:put does not check if peers are dialable, they are then stored / overridden with this new peer info, which has no listen address.
I then added an additional andalso to only store is_dialable peers, so that I had visiblity over the previous peerbook entry, and most of the time these were previously relayed devices, and the majority of the time the relayer is still contactable.
Sensecap M1 minors are experiencing the following issues since Firmware update 2021.09.03.0 has been implemented.
This is an across the board summary of issues.
Earnings on miners dropping an average of 50%
No beacons produced for 3-5 days, if produced, no witnesses being recorded. While other hotspots in the same area are working normally
Random re-sync of miners with no provocation, requiring a re-sync to firmware snap shot (now requiring miners to be down 6-7days after a full sync has already happened.)
Dual IP addresses with IP4 and p2p addresses occupying the same Helium Address line (rec-ode by Vagabond) but still continuing to show up
Miners going offline, with no user interaction on fully synced previously operational miners.
Most of this seems to be stemming from p2p interaction, and the API slowdown.
Multiple complaints from Helium Discord Hotspot-Issues and the Sensecap discord #troubleshooting channel have brought this here. Sensecap tech team has stated they are working with Helium to rectify this, but no official word from Helium acknowledging there is a problem or an ETA to fix these issues.
libp2p has session keepalives (for yamux). This sends an occasional message to keep the underlying connection open.
Just like ranch has a configurable acceptor limit, make the maximum number of streams configurable.
The go implementation of libp2p uses REUSEADDR to do TCP hole punching as described here:
http://www.bford.info/pub/net/p2pnat/index.html
This allows you to dial and listen on the same port, which means that, potentially, inbound connections to the NAT on the external port will end up back at the listener.
I think the key thing to change would be to make sure the swarm sets {reuseaddr, true} on the TCP sockets and that a dial on the swarm sets the {port, Port} option to the same port as the listener for the swarm is bound to. This should be a relatively simple change but, combined with STUN, should allow for more NAT traversals than simply relying on nat-upnp.
unsure if issue, but observed address is being reported by some peers as private ip address ranges, 192.168.x and 100.104.x - this setup is a bare metal dedicated server with fixed ip
2021-08-22 13:58:17.149 [info] <0.1244.0>@libp2p_transport_tcp:record_observed_addr:1003 peer "/p2p/11zh5nCB5sFQFmqFygAFuULAo1cPApDvkpj6EXFyRjATGyi7bze" informed us of our observed address "/ip4/<correct-static-public-ip>"
2021-08-22 13:58:17.902 [info] <0.1244.0>@libp2p_transport_tcp:record_observed_addr:1003 peer "/p2p/11B2HT4yTZdqfD8DSNyAyyGZZJhUnKdgZtACprr9PKd6B2Yen4Z" informed us of our observed address "/ip4/100.104.194.0/tcp/6803"
2021-08-22 13:58:18.052 [info] <0.1244.0>@libp2p_transport_tcp:record_observed_addr:1003 peer "/p2p/112CrVTut1g5gjBkHYBS911DdEsDtZg4WGH4RgdgkoLvtFzsjF5c" informed us of our observed address "/ip4/192.168.61.1/tcp/54906"
2021-08-22 13:58:18.052 [info] <0.1244.0>@libp2p_transport_tcp:record_observed_addr:1008 Saw 3 distinct observed addresses, assuming symmetric NAT
2021-08-22 13:58:18.481 [info] <0.1244.0>@libp2p_transport_tcp:attempt_port_forward_discovery:1040 dialed stungun peer "/p2p/112CrVTut1g5gjBkHYBS911DdEsDtZg4WGH4RgdgkoLvtFzsjF5c" looking for validation of "/ip4/192.168.61.1/tcp/44158"
2021-08-22 13:58:19.330 [info] <0.1244.0>@libp2p_transport_tcp:handle_info:584 Got dial back confirmation of observed address "/ip4/<correct-static-public-ip>"
2021-08-22 13:58:28.484 [info] <0.1244.0>@libp2p_transport_tcp:handle_info:525 stungun detected NAT type symmetric
Have the swarm (through a session_agent) maintain a configurable set of connections to randomly selected peers.
The configuration would ideally be a module name in the swam options with a simple default. So that applications can supply a more complex one if they have more context to be able to generate a diverse set of peer addresses from the peerbook.
When agent detects a disconnected peer connection it should randomly select a new peer address and connect to it.
Have the session_agent forget an existing connection on a regular schedule. It would then pick a new random address to connect to. The normal session timeout will end up closing the session.
For the set of open connections also be able to supply the list of stream handlers to start for a newly opened connection.
The simple policy uses a configuration list of seed_nodes passed in as part of #42, and will select a number of (non-stale) nodes from the peerbook and at least one from the list of nodes specified in seed_nodes.
@Vagabond does this capture what we had in mind?
I am trying this simple example:
-module(testapp1).
-export([echo/0]).
data_dir() ->
ok = filelib:ensure_dir("data"),
"data".
swarm(Name) ->
Opts = [{base_dir, data_dir()},
{libp2p_nat, [{enabled, false}]}],
{ok, Pid} = libp2p_swarm:start(Name, Opts),
ok = libp2p_swarm:listen(Pid, "/ip4/0.0.0.0/tcp/0"),
Pid.
echo() ->
Swarm1 = swarm(echo1),
Swarm2 = swarm(echo2),
timer:sleep(1000),
[_Addr1|_] = libp2p_swarm:listen_addrs(Swarm1),
[Addr2|_] = libp2p_swarm:listen_addrs(Swarm2),
{ok, Session} = libp2p_swarm:connect(Swarm1, Addr2),
timer:sleep(1000),
libp2p_swarm:stop(Swarm1),
libp2p_swarm:stop(Swarm2).
but I get the following warning messages:
1> testapp1:echo().
11:22:21.691 [info] libp2p_nat_server init with [#Ref<0.307704574.4151181315.127969>]
11:22:21.693 [info] libp2p_proxy_server init with [#Ref<0.307704574.4151181315.127969>,10]
11:22:21.703 [warning] nat is disabled
11:22:21.703 [info] Started Listener on ["/ip4/192.168.165.162/tcp/64001"]
11:22:21.713 [info] libp2p_nat_server init with [#Ref<0.307704574.4151181315.128256>]
11:22:21.713 [info] libp2p_proxy_server init with [#Ref<0.307704574.4151181315.128256>,10]
11:22:21.714 [warning] nat is disabled
11:22:21.714 [info] Started Listener on ["/ip4/192.168.165.162/tcp/64002"]
11:22:22.733 [notice] recording observed address "/p2p/112RuZ3qSUiBaQQyGXHyhdnpEnnWWPGveVfsmoS8pn5HKK9C9NZN" "/ip4/192.168.165.162/tcp/64001"
11:22:22.909 [warning] Failed to connect to "/p2p/11275kQ641uRGo1pkQR3VHGWboFeffb1exYt9G9vbdvYDuY9ytQ2": dialing_self
11:22:22.985 [warning] Failed to connect to "/p2p/112Y5XgyCJqp5dNYzeVECoHS3yMwGYa9FcsVatcDomLMihs48myG": dialing_self
11:22:23.977 [error] Session header read failed: enotconn Is this expected? Wat is this enotconn error?
When a libp2p node has bound to a listen address (with reuseaddr) and attempted nat-upnp, it needs to know if either strategy actually worked.
We can provide a STUN handler where dialing it tells you what the other side sees your IP/port as (apparently with an XOR on it so evil NATs don't rewrite the packet(?!)) and maybe you send some unique token. The other side then tries to dial your external IP/port and send you the token. If you get it, you know you have a hole punched and you can publish your external IP/port.
If that doesn't work, a TURN handler could be offered (on nodes with a working public address) where you can dial the peer, it gives you back a multiaddr like /turn/ip4/8.8.8.8/8888/SomeToken. Other peers can dial this and the TURN server can simply shuttle packets to the session it has with that NAT'd peer. The token allows a quick way to discard dials if they're for an already closed or invalid session.
I think the combination of these 2 strategies, along with #24 and #26, should allow most libp2p nodes to participate in the network.
https://en.wikipedia.org/wiki/STUN
https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT
https://en.wikipedia.org/wiki/Interactive_Connectivity_Establishment
crash log:
** exception exit: {{badarg,[{erlang,hd,[[]],[]},
{libp2p_swarm_server,listen_on,3,
[{file,"/Users/rahul/dev/fusion_blockchain/_build/default/lib/libp2p/src/libp2p_swarm_server.erl"},
{line,147}]},
{libp2p_swarm_server,handle_call,3,
[{file,"/Users/rahul/dev/fusion_blockchain/_build/default/lib/libp2p/src/libp2p_swarm_server.erl"},
{line,68}]},
{gen_server,try_handle_call,4,
[{file,"gen_server.erl"},{line,636}]},
{gen_server,handle_msg,6,
[{file,"gen_server.erl"},{line,665}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,247}]}]},
{gen_server,call,[<0.563.0>,{listen,"/ip6/::/tcp/0"}]}}
2> 13:30:08.327 [error] CRASH REPORT Process <0.563.0> with 0 neighbours crashed with reason: bad argument in call to erlang:hd([]) in libp2p_swarm_server:listen_on/3 line 147
13:30:08.327 [error] Supervisor {<0.560.0>,libp2p_swarm_sup} had child swarm_server started with libp2p_swarm_server:start_link(#Ref<0.3886377865.2429681665.90893>) at <0.563.0> exit with reason bad argument in call to erlang:hd([]) in libp2p_swarm_server:listen_on/3 line 147 in context child_terminated
13:30:08.328 [error] Lager event handler error_logger_lager_h exited with reason {'EXIT',{{case_clause,[<0.559.0>,undefined,exit,{{badarg,[{erlang,hd,[[]],[]},{libp2p_swarm_server,listen_on,3,[{file,"/Users/rahul/dev/fusion_blockchain/_build/default/lib/libp2p/src/libp2p_swarm_server.erl"},{line,147}]},{libp2p_swarm_server,handle_call,3,[{file,"/Users/rahul/dev/fusion_blockchain/_build/default/lib/libp2p/src/libp2p_swarm_server.erl"},{line,68}]},{gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,636}]},{gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,665}]},...]},...},...]},...}}
logger metadata allows events to be filtered and grouped. Things like peer address, stream id, handler, etc are all very helpful when debugging a live node.
Is there any way to let the node/miner listen to IPv6?
Currently a connect to IPv6 peer
<prog> peer connect /ip6/240e:xxxx:xxxx::xxx/tcp/44158
returns
Failed to connect to "/ip6/240e:xxxx:xxxx::xxx/tcp/44158"
running miner as validator, and etls on dedicated servers, using master libp2p, should be fixed single dedicated ip address, but getting stungun detected NAT type symmetric and also a random reports of ip address as 172. ipv4 network addresses, also incorrect.
several instances of this, but example:
2021-12-15 21:38:39.803 [info] <0.1280.0>@libp2p_transport_tcp:handle_info:500 retrying stungun with peer "/p2p/112iynLkeTKvPQnYCYhnrZp4sURMS77EHfCfsVUScG9PBmpf92U9"
2021-12-15 21:38:39.811 [info] <0.1280.0>@libp2p_transport_tcp:handle_info:527 stungun detected NAT type symmetric
2021-12-15 21:38:39.811 [info] <0.1356.0>@libp2p_relay_server:handle_cast:152 requested to init relay but we already have one @ <0.8525.1>
As observed by the community of validator operators, there are several validators and hotspots in the peer book with IP addresses in the 100.64.0.0/10 address. These are non-routable addresses in the shared address space as defined by RFC6598.
The is_public() check for IP addresses currently only considered RFC1918 address (10.0.0.0/8, 172.16.0.0/12, and 192.168.0.0). However there are several other non-routable IP addresses ranges. RFC6598 seems to be the next most common set apart from
The ets table name current is hardcoded as "config".
While ets allows multiple tables with the same name it'd be easier to tell swarms apart if we could name them.
Have swarm creation take a name or generate one if not given.
@Vagabond does this capture what we were chatting about?
Many EU ISPs change consumer IP addresses every 24-48 hours.
There have been a number of reports where the IP address changes, and while the API has a recent peerbook record the IP address was not updated to reflect the change either never, or it took a very long time which put it past the time for the next IP update.
Restarting the hotspot seems to fix it every time this was observed.
The theory is that IP change detection is failing, not being done often enough or not being done at all?
When a NAT is discovered, the nat server sends the transport listener a nat_discovered message. Sometimes, however, the message is not received. I do not know why this is.
edocs fail with the following error:
/Users/benoitc/BarrelProject/erlang-libp2p/src/peerbook/libp2p_peer.erl, function is_similar/2: at line 227: `-quote ended unexpectedly at line 229
edoc: skipping source file '/Users/benoitc/BarrelProject/erlang-libp2p/src/peerbook/libp2p_peer.erl': {'EXIT',
error}.
edoc: error in doclet 'edoc_doclet': {'EXIT',error}.
===> Failed to generate documentation for app 'libp2p'my understanding when reading the libo2p spec is that peerid can be used to manage users per nodes (each user has a key). How to create and add multiple peer ids / nodes? do you have any example for it?
often see gateways which are relayed through themselves, and compounding issue of other gateways relayed through a gateway which is relayed through itself, today there are quite a few:
examples:
/p2p/1129cq6aCeVkdPsu5AhtJHY3fg5FQzh8iJLzQ2eFiKmzWodn4WE6/p2p-circuit/p2p/1129cq6aCeVkdPsu5AhtJHY3fg5FQzh8iJLzQ2eFiKmzWodn4WE6
/p2p/11DFyMVWeDm5updfQ3ikCTRRMV5cp4UZaq17hyoDjcbY8m5Mxxz/p2p-circuit/p2p/11DFyMVWeDm5updfQ3ikCTRRMV5cp4UZaq17hyoDjcbY8m5Mxxz
/p2p/11SnMgBeoFhgQTSYBGqjDFtDYi2WH451KiJtw5JDJZ6zF9vCu33/p2p-circuit/p2p/11SnMgBeoFhgQTSYBGqjDFtDYi2WH451KiJtw5JDJZ6zF9vCu33
/p2p/11pdAcNwx9GJUsbsWWm5FCdhvRi35tMKg6i8d2UaLSoEf6w8j4X/p2p-circuit/p2p/11pdAcNwx9GJUsbsWWm5FCdhvRi35tMKg6i8d2UaLSoEf6w8j4X
/p2p/112WWx3PcX7LHrdSMVZvMkmgQSdCHfpsZvbWAaBT5JCKudVeesCM/p2p-circuit/p2p/112WWx3PcX7LHrdSMVZvMkmgQSdCHfpsZvbWAaBT5JCKudVeesCM
/p2p/112teifgjQT47W45CoJT5iE6MzbJpMTyoLawHqbTw6aeCcbF7q9t/p2p-circuit/p2p/112teifgjQT47W45CoJT5iE6MzbJpMTyoLawHqbTw6aeCcbF7q9t
/p2p/112kSbwTCWJw46PMjSXQhXK9ZV9ptd3inyUxXFYh18suw33PwWgH/p2p-circuit/p2p/112kSbwTCWJw46PMjSXQhXK9ZV9ptd3inyUxXFYh18suw33PwWgH
/p2p/112GoQNyTPAWriguTYsemZQHaGeFWQA4e1LZnhqjZWJNsqWpEaKx/p2p-circuit/p2p/112GoQNyTPAWriguTYsemZQHaGeFWQA4e1LZnhqjZWJNsqWpEaKx
/p2p/112Uet4q2xevRmF4bogj81R93NhcLbmh9zNfM2KsPuNw1AepZgiC/p2p-circuit/p2p/112Uet4q2xevRmF4bogj81R93NhcLbmh9zNfM2KsPuNw1AepZgiC
/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor/p2p-circuit/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor
multi cases, where one relayed gateway is relayed through a circular relay:
/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor/p2p-circuit/p2p/11JzkJF5tZHQqPX43BXhgT2uDKjqr9qAYiWgt6TGP12F6Eo825w
/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor/p2p-circuit/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor
/p2p/11pdAcNwx9GJUsbsWWm5FCdhvRi35tMKg6i8d2UaLSoEf6w8j4X/p2p-circuit/p2p/11cRVi41CYjMAyoJv6rat3CrYRg3pywH16CvXbTNuWPQz2C6EiL
/p2p/11pdAcNwx9GJUsbsWWm5FCdhvRi35tMKg6i8d2UaLSoEf6w8j4X/p2p-circuit/p2p/11pdAcNwx9GJUsbsWWm5FCdhvRi35tMKg6i8d2UaLSoEf6w8j4X
/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor/p2p-circuit/p2p/112BAE9KuJuZG8uaAfDJEvks3FF2jT8YA8TvN6nvW5a4gshFVVAk
/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor/p2p-circuit/p2p/11cSAzt2BmbDfshgGuo9teKTyBDnZkX3LUrZsTchLktARJnQsor
I would be nice to allow a user application to provide a callback for random dials to bias the dial. For example, blockchain core might want to export approximate location and use this callback to attempt to keep random dials on-continent.
After the 04.11.2021 update, there is no send beacons for 30 days. For solution, i changed the static ip to new 2 times, however there is no change. It started to getting annoying. How i can fix this problem or will there be uptade for fixing and when?
The peerbook will not accept stale peer entries, but after expiration time it will returns keys and values for stale entries.
I'd be nice to be able to define a maximum number of a particular kind of handler. Some handlers might require an expensive operation, or require expensive negotiation/authentication to establish and it'd be nice if there was a built in way to avoid them starting up if the maximum is reached.
Hello, there are at least 10-15 people on Sensecap discord reporting same issue that after 2021-09-03~ update
100% this change broke something for us: f31e45f
This is what we get via peerbook:
| address | name |listen_add|connectio| nat |last_updat|
+----------------------------------------------+-----------+----------+---------+-------+----------+
|/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5uj|odd-clay-wo| 2 | 5 |symmetr|15306.244s|
+----------------------------------------------+-----------+----------+---------+-------+----------+
+--------------------------------------------------------------------------------------------------+
| listen_addrs (prioritized) |
+--------------------------------------------------------------------------------------------------+
| /ip4/xxxxxxxxxxxxxx/tcp/44158 |
|/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7W|
+--------------------------------------------------------------------------------------------------+
+---------------------------------------------------------+
| connections |
+---------------------------------------------------------+
|/p2p/11KgYQR9h3Rz9cmzVZLwtZiHxThMs731uFTraLiyPmfHcxp7DYx |
|/p2p/1128k7YK3UfSh5qaAp37qe2jw3LaG6ycQUAAqD33L9v9DoRHdrkK|
|/p2p/112BwJXdn117Y9pNcte1VKpZWFViakbi9tuUMrDsmpnZsM6DEfPb|
|/p2p/11GxM19uKgCpoArMGBUDGeNLxRYM8L9ferCCd8sHkgFdZtVU86P |
|/p2p/11XG5Dw6L1jmu9XE61b1LJ6TdgWm7An1a1Nut34XwMbUyiUWoQV |
+---------------------------------------------------------+
Sensecap dashboard reporting this:
Listen IP(Based on Helium API):
/ip4/xxxxxxxx/tcp/44158,/p2p/1121Gc63SGenk9emVaEe3TsQejLgQHp193ptawGmtp7q34M4X9yQ/p2p-circuit/p2p/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8
NAT Type: Symmetric
Relayed: Yes
Port 44158 has been confirmed to be open as per https://www.yougetsignal.com/tools/open-ports/
The hotspot WAS NOT relayed until last update!
The NAT type on Router is COG (In other words I presume CONE), NOT SYMMETRIC! And this change somehow changes our nat type to symmetric, but it's not symmetric!
https://explorer.helium.com/hotspots/113gAgf8DTFi4YQFrQYMRhN7WUj25rTwS2mkTM5ujXP68mCGNw8/activity
HELIUM EXPLORER DOES NOT REPORT THAT HOTSPOT IS RELAYED!
After we reboot the hotspot, the relay is reporting is NONE and Relayed status is: No.
After 2-4 hours, poof, we become relayed and start listing on both IP + P2P, which should not happen.
I own 3 hotspots, 1 hotspot on same ISP, does not have this type of issue, which is super odd, but this can't because of our router settings as technically we are not choosing to somehow assign 2 listening IPs to API, which should not be happening.
Please help solve this issue ASAP!
We should probably thread some kind of identify through stungun to that if we encounter a different host that we detect that properly.
We need one, ideally compatible with Go libp2p or ipfs.
Have a session timeout after a configurable time (default 60s?) has elapsed without any active streams.
Part of #43 to have sessions close when they're forgotten
currently getting a stale peer is counted against the number of tries, and it shouldn't be, as when the network is badly degraded the chance of getting a sync peer gets lower and lower, making things even worse.
DHCP based nodes will change their IP address over time.
Have the tcp transport server poll for IP address changes and update ETS as well as notify the peerbook
Take an Options map in lib2p_swarm:start and use instead of application:get_env to fetch configuration values.
This will make external configuration of libp2p as a library easier.
The following options should be supported with sensible built-in defaults:
Being able to just get an identify record back from a remote swarm will
Currently, due to a suspected ordering/determinism issue in deserialization and reserialization, gossip metadata becomes unstable once it has more than four or five entries.
In order to report richer information about hotspots, it would be nice to fix this issue so we could report more and better information.
is there anything missing for that transport compared to the spec?
Can a transport only be added after a swarm is started? Is this supported or something is unfinished?
Experiencing a weird issue where my listen_addrs is changing to a random IP address. Here's my RAKv2 miner api
https://api.helium.io/v1/hotspots/name/rough-fuzzy-weasel
My real IP is 198.47.45.143 and I've completed the port forwarding, DHCP reservation, UPNP disabling. Tests confirm it is open. ISP IP is dynamic but it hasn't changed at all.
But for some reason the API is showing someone else's public IP
"listen_addrs": [
"/ip4/70.189.158.93/tcp/44158"
],
70.189.158.93 is not mine.
This also happened just over 24 hours ago with the IP 67.241.40.128, but when you do a port 44158 test on this IP the port is open. So it looks like it's another miners IP.
I spoke to BFGNeil on discord and he helped to check it out, and he tried the following:
A few variables i should mention.
Can anyone help to determine why I'm getting these random IP's and what I can do to stop gossiping a wrong IP address?
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
Django
The Web framework for perfectionists with deadlines.
Laravel
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
Microsoft
Open source projects and samples from Microsoft.
Google
Google ❤️ Open Source for everyone.
Alibaba
Alibaba Open Source for everyone
D3
Data-Driven Documents codes.
Tencent
China tencent open source team.