The account has been initialized in aptos cli and can be seen in the config.yaml, but the execution of the command returned an error:

aptos account list --profile ait3-operator

and, it is normal when I use this command:

aptos account list --account 76bda37202dce72918b4e4170024a16503156cd602360b1af470dda8c5dee4f1

Node stop synchronizing after 15 epoch, restarting , updating , deleting db dont helps

owner address
0x337925eaba30d718f87ddc251c8c3314322f8e0bb87cf2e6c77cc4211b97e493

aptos_state_sync_version{type="applied_transaction_outputs"} 66202840
aptos_state_sync_version{type="executed_transactions"} 65533161
aptos_state_sync_version{type="synced"} 66202840
aptos_state_sync_version{type="synced_epoch"} 32

aptos_connections{direction="inbound",network_id="Validator",peer_id="337925ea",role_type="validator"} 209
aptos_connections{direction="outbound",network_id="Validator",peer_id="337925ea",role_type="validator"} 7

log.txt

The default Ubuntu Server 20.04 LTS operating system file descriptor limit ulimit is just 1024.

This governs file access, and I believe it is the cause of the following error:

I set the file descriptor limit persistently with the following steps:

increase file descriptors

ulimit -n 500000

make limits permanent

sudo nano /etc/security/limits.conf

append to the end of the limits.conf. replace yourusername with the output from whoami (node user)

yourusername soft nproc 500000
yourusername soft nproc 500000
yourusername hard nproc 500000
yourusername soft nofile 500000

verify your file handlers have been increased

ulimit -n

for a long time, nodes with ip
179.48.251.14
66.94.104.49
34.134.37.36
80.211.33.21
13.246.149.59
44.242.152.37
92.255.176.91
95.188.77.87
34.95.49.24
are inactive (and probably will not be active anymore)...
connecting to them causes additional (useless) load on the network

is there any way to block these peers (at least manually)??

@sherry-x
Validator node is stopping and needs restart after a couple of hours

Log is attached.
Thanks
log2_stopping.txt

Hello! owner address is: 0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764

Metrics:

aptos_connections{direction="inbound",network_id="Validator",peer_id="168625c7",role_type="validator"} 67
aptos_connections{direction="inbound",network_id="vfn",peer_id="00000000",role_type="validator"} 1
aptos_connections{direction="outbound",network_id="Validator",peer_id="168625c7",role_type="validator"} 151
aptos_connections{direction="outbound",network_id="vfn",peer_id="00000000",role_type="validator"} 0
aptos_consensus_proposals_count 7
aptos_state_sync_version{type="synced"} 75916080

validator_epoch_history:

{
"epoch": 30,
"failed_proposal_count": 2,
"max_potential_reward": 1598296064,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764",
"voting_power": 100012268583535
},
{
"epoch": 31,
"failed_proposal_count": 3,
"max_potential_reward": 1598296064,
"rewards_earned": 998935040,
"successful_proposal_count": 5,
"validator_addr": "0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764",
"voting_power": 100012268583535
},
{
"epoch": 32,
"failed_proposal_count": 2,
"max_potential_reward": 1598312028,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764",
"voting_power": 100013267518575
},
{
"epoch": 33,
"failed_proposal_count": 2,
"max_potential_reward": 1598312028,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764",
"voting_power": 100013267518575
},

Failed with error message "Waiting for transaction 0xebebed8756bb63804d0a4256e6588a46a6526a62d8b39c9bc3797b89d9eb8df3 timed out!". Please try again.

Owner address:
0xb0cc635492330102e2e0592033045f0b14381a9cbd8f43f21223fc1fa3b58ac6

My node stop synchronizing constantly. after epoch 25, proposals drops quickly.

log is here.

log.txt

Recently, it has been observed that the consensus will stop when the program runs for a period of time, and the blocks are not synchronized. The detection node process is normal, and finally the node has to be restarted to return to normal.

my address:
https://explorer.devnet.aptos.dev/account/0x50c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e?network=ait3

testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:54.144813Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"0a5d60ca5c94b53c75dfc19d8d1ecc624b10565143ca90199c071089eb86173c"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:54.438314Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"57be1bdaf544178f3e754dd23f84c74a95785c694066f6c1d48762b8c888cbee"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:54.612853Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"da26d0b97726a315035486da1562a1e7d42c1f8bd50fee123402a1e5d4e3af92"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"network","file":"network/src/peer_manager/transport.rs:309"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:54.705148Z","message":"[validator,Validator,77700fe6] Inbound connection from /ip4/18.230.78.80/tcp/39814 failed to upgrade after 0.000 secs: noise server: client 351a3e91: client connecting with unauthenticated peer id: 351a3e91731204baa596e094366152a77b8d4454065fdc7eff1c6879bddf48a6","data":{"error":"noise server: client 351a3e91: client connecting with unauthenticated peer id: 351a3e91731204baa596e094366152a77b8d4454065fdc7eff1c6879bddf48a6","network_address":"/ip4/18.230.78.80/tcp/39814","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"}}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:54.847414Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"5eea1639b99765ff96502180949fac3662ffd89d358e4adff5dcd31b60ed72c0"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:55.118016Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"699b1fe83796492513a17f276607a4d3dbc250ac26e9d5f455ea790776b6337e"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:55.381118Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"2ec5fe60d0e96c4791826635be764ad9e24db9918642aff078244dcb03b24203"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:55.647184Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"dc10d8a940d909cd3c26b8d2c0378d6aab4165b5d98a1f5bdc592aeaa6cdca5f"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:55.708826Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"be2ac53a4dba0325cf3d65eca93c68e5239c6fc5b4ca013e6ec7089235c43775"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:55.987909Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"8485b0e4779190c2e0cefc899ac853cd3d878930e66587df8d867c0c941267a6"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:56.260114Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"fcfa1efb940e23936915c2a6814ca557dbf3c6972d4ff0154d8e6aacda905c64"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:56.299465Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"0289311ada457c33abfaaabff61177c935a3705ac7ac624a43e767615cad2548"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:56.532155Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"e38cec9883de5eaa58452d4860e13efabeb986876a7e51cc0e055f540f1a9bf4"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:56.840492Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"c9b19c9b6afb63f3e9fd8c0491fa6ca9fa6a008cd5f2b2dc3b5d7a984b9c5a7f"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.140782Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"b7aa84d8be549026574559735403a79b001ca2c3be54d405b8ec7c4df79c1b15"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.590372Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"08c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.705525Z","message":"[validator,Validator,77700fe6] Dialing peer 66bdb52e at /ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0","data":{"network_address":"/ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"66bdb52ecd77bace3b5ae4c014b69d0a29957f27b3dcb7406c806b4052e05f4c"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.713235Z","message":"[validator,Validator,77700fe6] Dialing peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","data":{"network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.794287Z","message":"[validator,Validator,77700fe6] Dialing peer e10db9a9 at /ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0","data":{"network_address":"/ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"e10db9a98a2351837475fc89e3e7adaa69a6e4f83c78b2948ebcf5e0551446da"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.795389Z","message":"[validator,Validator,77700fe6] Dialing peer 614d8a2a at /ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0","data":{"network_address":"/ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"614d8a2a081e4e51bd216648ec4fafd6c3d0ff03f6f7e83a044b989c738438cf"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:57.865688Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"b1bc346da7756a51dae71ddbd8d74228604598fa6cc35a52fb78e86ca3915e26"}}
testnet-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.038043Z","message":"[validator,Validator,77700fe6] Outbound connection failed for peer e10db9a9 at /ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"e10db9a98a2351837475fc89e3e7adaa69a6e4f83c78b2948ebcf5e0551446da"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.038102Z","message":"[validator,Validator,77700fe6] Failed to connect to peer: e10db9a9 at address: /ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"e10db9a98a2351837475fc89e3e7adaa69a6e4f83c78b2948ebcf5e0551446da"}}
testnet-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.146243Z","message":"[validator,Validator,77700fe6] Outbound connection failed for peer 66bdb52e at /ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"66bdb52ecd77bace3b5ae4c014b69d0a29957f27b3dcb7406c806b4052e05f4c"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.146306Z","message":"[validator,Validator,77700fe6] Failed to connect to peer: 66bdb52e at address: /ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/90.188.13.9/tcp/6180/noise-ik/0x5d76d1c24e35383f9f46883c77ca9f0cfcec368a6e24abafc9a00eb5a3d0b26c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"66bdb52ecd77bace3b5ae4c014b69d0a29957f27b3dcb7406c806b4052e05f4c"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.160070Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"8aa4d82016b1a0356421c22cda47990d8ca5ffc0f5a012754817db871d34ae90"}}
testnet-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.189770Z","message":"[validator,Validator,77700fe6] Outbound connection failed for peer 614d8a2a at /ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"614d8a2a081e4e51bd216648ec4fafd6c3d0ff03f6f7e83a044b989c738438cf"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.189841Z","message":"[validator,Validator,77700fe6] Failed to connect to peer: 614d8a2a at address: /ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"614d8a2a081e4e51bd216648ec4fafd6c3d0ff03f6f7e83a044b989c738438cf"}}
testnet-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.249600Z","message":"[validator,Validator,77700fe6] Outbound connection failed for peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.249684Z","message":"[validator,Validator,77700fe6] Failed to connect to peer: 3a9bcf67 at address: /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.532757Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"f7c045f174d8527bd300eb5e9e84fb6db43a9e80190f7b9597db095fa0995658"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:359"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.832255Z","message":"[validator,Validator,77700fe6] New connection established: [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/154.26.128.150/tcp/51898/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":276,"connection_origin":"inbound","network_address":"/ip4/154.26.128.150/tcp/51898/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:645"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.832286Z","message":"[validator,Validator,77700fe6] Closing incoming connection with Peer 28dde00b to mitigate simultaneous dial","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:58.837071Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"0e41c1afa2edc48280f24f0b456311086fa3f777f25b2db7d9d12a91526214a2"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:59.106129Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"5badd632abd92d673892e42578151881d644472326b6c021603f1110f10f74b1"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:59.173932Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"69268e38beda87f1ff5bf22881f22b9deb564c89028e7af89ee8b911fc616f95"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:59.573586Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"b12cf85a0a28c36b5703ff9aa5fcc995a3c64f8869f68f61ffbc17b1b6814d25"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:45:59.726991Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"873cc8a29a8ad6afe1d13ffbaa512a91be22bb1b513ac4bd0b1077200d6be96f"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:00.135454Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:00.503518Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"e0aff63df5c4b9ce3d7e415e2c0b8aa2f036f0a0d730ab31546bf45a2569d925"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.504870Z","message":"Err(Rpc timed out), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"26a5f68e39a2543aef7ceaec7dcd047510d856b495f95979879bd278e4725e07"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.504890Z","message":"[validator,Validator,77700fe6] Error making outbound rpc request with request_id 561 to 26a5f68e: Error: Rpc timed out","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"26a5f68e39a2543aef7ceaec7dcd047510d856b495f95979879bd278e4725e07"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.592781Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"23f422eb4212100c47cdc28c242cab052b65ace49d1429c1197f4a1b153b178e"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.694625Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"0f33a11bc2cc943b451e17e4d74dfc7fcc3af2bf5a6ba6dfefffc4f7c7e95d04"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.857074Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"e586f8a8c30b8cc949b9642c5c8d41548dc4558385656e46cd5b5be45a2c9d2a"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"network","file":"network/src/peer_manager/transport.rs:309"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:02.940643Z","message":"[validator,Validator,77700fe6] Inbound connection from /ip4/66.94.104.49/tcp/41056 failed to upgrade after 0.048 secs: noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376","data":{"error":"noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376","network_address":"/ip4/66.94.104.49/tcp/41056","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"}}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.107294Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"ae91b5f2c5a13f5394c46bcfc7e67e4db1b3659dc6c6faa08e4b9a76a8dde53d"}}
testnet-validator-1 | {"level":"ERROR","source":{"package":"consensus","file":"consensus/src/round_manager.rs:803"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.107385Z","data":{"committed_round":0,"error":"from peer 631bc1f429ca72def1ebb108c442211b6a2e9337188342ef7b1eb132aa91efc4\n\nCaused by:\n 0: [RoundManager] Stop processing vote\n 1: Failed to fetch block 5800c898 in 133 attempts: no more peers available","kind":"InternalError","pending_votes":"PendingVotes: [LI dc8804a6 has 4 votes [03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824, 28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186, 46774d7bda97925b0f7dc7eecc927fe4f2f17739f087243043694d2bea153cf5, 6ebbe079ff81335cb9b1ab2c667a15c9994b60001768b6a2236b19b4c273b71d] 4 timeout [03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824, 28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186, 46774d7bda97925b0f7dc7eecc927fe4f2f17739f087243043694d2bea153cf5, 6ebbe079ff81335cb9b1ab2c667a15c9994b60001768b6a2236b19b4c273b71d]]","round":1}}
testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.107430Z","message":"Local state SyncInfo[certified_round: 0, ordered_round: 0, timeout round: 0, commit_info: BlockInfo: [epoch: 29, round: 0, id: 560de343, executed_state_id: 6d828a98, version: 55339297, timestamp (us): 1661983411050037, next_epoch_state: None]], remote state SyncInfo[certified_round: 8692, ordered_round: 8691, timeout round: 0, commit_info: BlockInfo: [epoch: 29, round: 8691, id: 2b48c67c, executed_state_id: 0d4d0e25, version: 58296665, timestamp (us): 1661989290390899, next_epoch_state: None]]","data":{"epoch":29,"event":"ReceiveNewCertificate","remote_peer":"0289311ada457c33abfaaabff61177c935a3705ac7ac624a43e767615cad2548","round":1}}
testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:208"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.110391Z","message":"Start state sync to commit cert: QuorumCert: [VoteData: [block id: 5800c898, epoch: 29, round: 8692, timestamp: 1661989290979765,parent_block_id: 2b48c67c, parent_block_round: 8691, parent_timestamp: 1661989290390899], LedgerInfo: [commit_info: BlockInfo: [epoch: 29, round: 8691, id: 2b48c67c, executed_state_id: 0d4d0e25, version: 58296665, timestamp (us): 1661989290390899, next_epoch_state: None]]], ordered cert: QuorumCert: [VoteData: [block id: 5800c898, epoch: 29, round: 8692, timestamp: 1661989290979765,parent_block_id: 2b48c67c, parent_block_round: 8691, parent_timestamp: 1661989290390899], LedgerInfo: [commit_info: BlockInfo: [epoch: 29, round: 8691, id: 2b48c67c, executed_state_id: 41434355, version: 0, timestamp (us): 1661989290390899, next_epoch_state: None]]]","data":{"event":"StateSync","remote_peer":"0289311ada457c33abfaaabff61177c935a3705ac7ac624a43e767615cad2548"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:422"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.110412Z","message":"Retrieving 2 blocks starting from 5800c898"}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.153404Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"0289311ada457c33abfaaabff61177c935a3705ac7ac624a43e767615cad2548"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.400620Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"19105a85b8d455e51a1b0f917dbb2464d47c47a325c20e5aaa64916299e06aa4"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:359"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.830786Z","message":"[validator,Validator,77700fe6] New connection established: [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/154.26.128.150/tcp/52082/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":277,"connection_origin":"inbound","network_address":"/ip4/154.26.128.150/tcp/52082/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:645"},"thread_name":"network-Validator","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.830818Z","message":"[validator,Validator,77700fe6] Closing incoming connection with Peer 28dde00b to mitigate simultaneous dial","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:03.849294Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"d801e1b2c79395df00ff26f63f60e2f11d541d18d6755831ec1a6537fd3ad6d3"}}
testnet-validator-1 | {"level":"WARN","source":{"package":"consensus","file":"consensus/src/block_storage/sync_manager.rs:482"},"thread_name":"consensus","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:04.080609Z","message":"Ok(BlockRetrievalResponse { status: IdNotFound, blocks: [] }), Failed to fetch block, trying another peer","data":{"block_id":"5800c89819a1540bf1f90acf3e6c59390330441cf81bafd2dc662d9c33fb931c","remote_peer":"bd2d5cb253a3ae97ee84039b34ec753c3bca038a41ab953b9f6df4ac38bf7d35"}}
testnet-validator-1 | {"level":"INFO","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:728"},"thread_name":"aptos-data-client","hostname":"4d63246a5c79","timestamp":"2022-09-01T00:46:04.098664Z","data":{"event":"priority_and_regular_peers","message":"Number of priority peers: 215. Number of regular peers: 0","name":"peer_states"}}

I used wallet version v0.1.8 on Chrome
I signed in with my wallet address: 0x1614c0510f246f089e63cb0dbf1c30529d6389d4b6591d209afd821065bf3541
I connected my wallet at https://explorer.devnet.aptos.dev/proposals/staking?network=ait3
Completed the form as follows:

Staking Amount: 100000000000000
Operator Address: 8835cbc88a6126b97e5df597b45bef22498738810ad381efdab7bb518b632432
Voter Address: 0x1614c0510f246f089e63cb0dbf1c30529d6389d4b6591d209afd821065bf3541

Hit Submit button then i get:

Failed with error message "Invalid transaction: Type: Validation Code: TRANSACTION_EXPIRED". Please try again.

Trying to sync with inactive nodes will likely stop the sync process

-- Logs begin at Wed 2022-08-31 17:15:39 UTC. --
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:43.922576Z WARN crates/inspection-service/src/lib.rs:37 Metric Family 'aptos_state_sync_epoch_state_gauges' over 1000 dimensions '3755' {"count":3755,"metric_family":"aptos_state_sync_epoch_state_gauges"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.196076Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:582 [validator,Validator,b1a242d8] Dialing peer 49bbd1c4 at /ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0 {"network_address":"/ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"49bbd1c4b5313d97a9218d97d48020392d3991e978886802d51d36bb747f7bf1"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.224212Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:582 [validator,Validator,b1a242d8] Dialing peer 95ce20e7 at /ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0 {"network_address":"/ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"95ce20e79294e79fca729d40db823144c677a27eec7a53c5cd021dd28602c6d7"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.259838Z [network-Validator] ERROR network/src/peer_manager/transport.rs:258 [validator,Validator,b1a242d8] Outbound connection failed for peer 49bbd1c4 at /ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"49bbd1c4b5313d97a9218d97d48020392d3991e978886802d51d36bb747f7bf1"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.259948Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:869 [validator,Validator,b1a242d8] Failed to connect to peer: 49bbd1c4 at address: /ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/194.5.152.243/tcp/6180/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"49bbd1c4b5313d97a9218d97d48020392d3991e978886802d51d36bb747f7bf1"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.290976Z [network-Validator] ERROR network/src/peer_manager/transport.rs:258 [validator,Validator,b1a242d8] Outbound connection failed for peer 95ce20e7 at /ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"95ce20e79294e79fca729d40db823144c677a27eec7a53c5cd021dd28602c6d7"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:44.291107Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:869 [validator,Validator,b1a242d8] Failed to connect to peer: 95ce20e7 at address: /ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/194.5.152.240/tcp/6180/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"95ce20e79294e79fca729d40db823144c677a27eec7a53c5cd021dd28602c6d7"}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:45.965709Z [network-Validator] WARN network/src/peer_manager/transport.rs:309 [validator,Validator,b1a242d8] Inbound connection from /ip4/78.27.202.227/tcp/44346 failed to upgrade after 0.000 secs: noise server: error reading client handshake init message: unexpected end of file {"error":"noise server: error reading client handshake init message: unexpected end of file","network_address":"/ip4/78.27.202.227/tcp/44346","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"}}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:47.706603Z [network-Validator] WARN network/src/peer_manager/transport.rs:309 [validator,Validator,b1a242d8] Inbound connection from /ip4/66.94.104.49/tcp/52262 failed to upgrade after 0.000 secs: noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376 {"error":"noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376","network_address":"/ip4/66.94.104.49/tcp/52262","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"}}
Aug 31 22:44:49 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:49.141108Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:582 [validator,Validator,b1a242d8] Dialing peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0 {"network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:49.390797Z [network-Validator] ERROR network/src/peer_manager/transport.rs:258 [validator,Validator,b1a242d8] Outbound connection failed for peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:49.390924Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:869 [validator,Validator,b1a242d8] Failed to connect to peer: 3a9bcf67 at address: /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:50.885731Z WARN crates/inspection-service/src/lib.rs:37 Metric Family 'aptos_state_sync_epoch_state_gauges' over 1000 dimensions '3755' {"count":3755,"metric_family":"aptos_state_sync_epoch_state_gauges"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:51.037902Z [aptos-data-client] INFO state-sync/aptos-data-client/src/aptosnet/mod.rs:805 {"event":"aggregate_summary","message":"Global data summary: AdvertisedData { Highest epoch ending ledger info, epoch: Some(28). Highest synced ledger info, epoch: Some(29), version: Some(56584955). Lowest transaction version: Some(0), Lowest transaction output version: Some(0), Lowest states version: Some(534412) }, OptimalChunkSizes { epoch_chunk_size: 100, state_chunk_size: 2000, transaction_chunk_size: 2000, transaction_output_chunk_size: 2000 }","name":"peer_states"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:51.134119Z [aptos-data-client] INFO state-sync/aptos-data-client/src/aptosnet/mod.rs:728 {"event":"priority_and_regular_peers","message":"Number of priority peers: 212. Number of regular peers: 1","name":"peer_states"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:52.027108Z [network-Validator] WARN network/src/peer_manager/transport.rs:309 [validator,Validator,b1a242d8] Inbound connection from /ip4/78.27.202.227/tcp/45284 failed to upgrade after 0.000 secs: noise server: error reading client handshake init message: unexpected end of file {"error":"noise server: error reading client handshake init message: unexpected end of file","network_address":"/ip4/78.27.202.227/tcp/45284","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"}}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:54.140659Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:582 [validator,Validator,b1a242d8] Dialing peer 28dde00b at /ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0 {"network_address":"/ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:54.366322Z [network-Validator] ERROR network/src/peer_manager/transport.rs:258 [validator,Validator,b1a242d8] Outbound connection failed for peer 28dde00b at /ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:54.366471Z [network-Validator] INFO network/src/connectivity_manager/mod.rs:869 [validator,Validator,b1a242d8] Failed to connect to peer: 28dde00b at address: /ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file {"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/5.161.137.197/tcp/6180/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:57.889031Z WARN crates/inspection-service/src/lib.rs:37 Metric Family 'aptos_state_sync_epoch_state_gauges' over 1000 dimensions '3755' {"count":3755,"metric_family":"aptos_state_sync_epoch_state_gauges"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:44:57.960779Z [network-Validator] WARN network/src/peer_manager/transport.rs:309 [validator,Validator,b1a242d8] Inbound connection from /ip4/78.27.202.227/tcp/46250 failed to upgrade after 0.000 secs: noise server: error reading client handshake init message: unexpected end of file {"error":"noise server: error reading client handshake init message: unexpected end of file","network_address":"/ip4/78.27.202.227/tcp/46250","network_context":{"role":"validator","network_id":"Validator","peer_id":"b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988"}}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:45:01.134126Z [aptos-data-client] INFO state-sync/aptos-data-client/src/aptosnet/mod.rs:728 {"event":"priority_and_regular_peers","message":"Number of priority peers: 212. Number of regular peers: 1","name":"peer_states"}
Aug 31 22:45:03 HTZ3TTR-AFRONE aptos-node[6382]: 2022-08-31T22:45:01.136500Z [aptos-data-client] INFO state-sync/aptos-data-client/src/aptosnet/mod.rs:805 {"event":"aggregate_summary","message":"Global data summary: AdvertisedData { Highest epoch ending ledger info, epoch: Some(28). Highest synced ledger info, epoch: Some(29), version: Some(56590079). Lowest transaction version: Some(0), Lowest transaction output version: Some(0), Lowest states version: Some(534412) }, OptimalChunkSizes { epoch_chunk_size: 100, state_chunk_size: 2000, transaction_chunk_size: 2000, transaction_output_chunk_size: 2000 }","name":"peer_states"}

my node address:

https://explorer.devnet.aptos.dev/account/0x50c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e?network=ait3

this is error logs:

2022-09-01T07:00:01.822447931Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:01.586364Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer c3f2c9dc at /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
2022-09-01T07:00:02.628638418Z {"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:434"},"thread_name":"aptos-data-client","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:02.481801Z","data":{"error":"UnexpectedErrorEncountered("Application layer unexpectedly dropped response channel")","event":"response_error","name":"storage_service_response","peer":"Validator:33e6210e","request_id":179029,"request_type":"get_storage_server_summary_compressed"}}
2022-09-01T07:00:02.628678038Z {"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:788"},"thread_name":"aptos-data-client","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:02.481830Z","data":{"error":"UnexpectedErrorEncountered("Application layer unexpectedly dropped response channel")","event":"peer_polling_error","message":"Error encountered when polling peer!","name":"storage_summary_response","peer":"Validator:33e6210e"}}
2022-09-01T07:00:05.718000153Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:05.524755Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer b77026ce at /ip4/3.133.88.245/tcp/6180/noise-ik/0x3829ed5aabd623fdb814ff22a62293630f1736dfb7a5d5fddcdf8fe4c59e892c/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/3.133.88.245/tcp/6180/noise-ik/0x3829ed5aabd623fdb814ff22a62293630f1736dfb7a5d5fddcdf8fe4c59e892c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"b77026ce272a63b7261d20e5d0d9ca8cddd81b42b3432891668c43c03dbd1b73"}}
2022-09-01T07:00:06.576848998Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:06.523506Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer e10db9a9 at /ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/149.102.134.38/tcp/6180/noise-ik/0xb2b4c8e5011535b30fc1444a8a19a471e33a502a78a448a5a53a9866bb885a2e/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"e10db9a98a2351837475fc89e3e7adaa69a6e4f83c78b2948ebcf5e0551446da"}}
2022-09-01T07:00:06.576863906Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:06.576445Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 614d8a2a at /ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/185.173.157.251/tcp/6180/noise-ik/0x55a9a93048966c2673caffd454058b96bb86448f7486c12453763cdd0e9fd623/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"614d8a2a081e4e51bd216648ec4fafd6c3d0ff03f6f7e83a044b989c738438cf"}}
2022-09-01T07:00:07.031206513Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:06.752994Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
2022-09-01T07:00:08.322084156Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:08.267928Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 0851fbcc at /ip4/35.232.235.205/tcp/6180/noise-ik/0xb1251f6b9f8c7d25ffae475297da56ede376d192e82983e668134b1525abd151/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/35.232.235.205/tcp/6180/noise-ik/0xb1251f6b9f8c7d25ffae475297da56ede376d192e82983e668134b1525abd151/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"0851fbcc396e403f8f910870f05d4307b275f73f96cfaaae2c06ca0b93aa77b5"}}
2022-09-01T07:00:08.322126077Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:08.282793Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer ed8f98dd at /ip4/92.101.121.75/tcp/6080/noise-ik/0xd12bb15376cb42a244d082f8b904b72833d787fd24d743d2cb8b2cdc351fba6a/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/92.101.121.75/tcp/6080/noise-ik/0xd12bb15376cb42a244d082f8b904b72833d787fd24d743d2cb8b2cdc351fba6a/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"ed8f98dd4c607ea196fdd0edb9aafa68e7fe31f9eb736f93e9dc744954d960e5"}}
2022-09-01T07:00:08.322158823Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:08.321887Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 45cf704e at /ip4/73.158.251.45/tcp/6180/noise-ik/0xc06f9d8d2ea6759a18ae2828817114db3df4f554f761f3b9cdd2d4ae89ee654a/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/73.158.251.45/tcp/6180/noise-ik/0xc06f9d8d2ea6759a18ae2828817114db3df4f554f761f3b9cdd2d4ae89ee654a/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"45cf704e3552993db28b0235fec04bc279b659f3eb64269f93f2f7a6c9bd8ce0"}}
2022-09-01T07:00:08.792985070Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:08.537970Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 7b689638 at /ip4/192.81.211.59/tcp/6180/noise-ik/0x792c7a163ed9787b0340461cd9968695e914ddc9522d8f95d78f9300f85ae656/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/192.81.211.59/tcp/6180/noise-ik/0x792c7a163ed9787b0340461cd9968695e914ddc9522d8f95d78f9300f85ae656/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"7b689638b3a5473309f49325f87284ffca126e126f80495ad6c185e43eecb7d3"}}
2022-09-01T07:00:10.229069217Z {"level":"ERROR","source":{"package":"consensus","file":"consensus/src/network_interface.rs:188"},"thread_name":"consensus","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:10.218917Z","message":"Unavailable peers: [b77026ce272a63b7261d20e5d0d9ca8cddd81b42b3432891668c43c03dbd1b73, 614d8a2a081e4e51bd216648ec4fafd6c3d0ff03f6f7e83a044b989c738438cf, 3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0, c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1, 1e04892640120d294c35cc85033634af8e93c04574771c1335d4ea6d89dbfb6e, 352eb70e6ae3f13fd132bb56a79ddaa8b1ce41e4f26f3a8dc058c737728dc474, e10db9a98a2351837475fc89e3e7adaa69a6e4f83c78b2948ebcf5e0551446da]"}
2022-09-01T07:00:10.229199306Z {"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:434"},"thread_name":"aptos-data-client","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:10.228683Z","data":{"error":"TimeoutWaitingForResponse("Rpc timed out")","event":"response_error","name":"storage_service_response","peer":"Validator:0d8fee82","request_id":179048,"request_type":"get_storage_server_summary_compressed"}}
2022-09-01T07:00:10.229240801Z {"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:788"},"thread_name":"aptos-data-client","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:10.228717Z","data":{"error":"TimeoutWaitingForResponse("Rpc timed out")","event":"peer_polling_error","message":"Error encountered when polling peer!","name":"storage_summary_response","peer":"Validator:0d8fee82"}}
2022-09-01T07:00:11.988120870Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:11.942363Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 352eb70e at /ip4/18.230.78.80/tcp/6180/noise-ik/0x30c27955d6a4473c56077b7cb716d9fb0aae68bf01aa93870ea83194860c8f6d/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/18.230.78.80/tcp/6180/noise-ik/0x30c27955d6a4473c56077b7cb716d9fb0aae68bf01aa93870ea83194860c8f6d/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"352eb70e6ae3f13fd132bb56a79ddaa8b1ce41e4f26f3a8dc058c737728dc474"}}
2022-09-01T07:00:13.659696529Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:13.294491Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer eb15bc1d at /dns/ait3.astrader.net/tcp/6180/noise-ik/0x506064e642b017a297675314a33ad8316cc581ee40dde39c8bdcd04f9053a853/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/dns/ait3.astrader.net/tcp/6180/noise-ik/0x506064e642b017a297675314a33ad8316cc581ee40dde39c8bdcd04f9053a853/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"eb15bc1d36806e5079d8699c404e3aba86cb136751be943969d6373570c88b7e"}}
2022-09-01T07:00:13.659708522Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:13.319078Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 5ee6ee04 at /ip4/44.242.152.37/tcp/6180/noise-ik/0x614912d50b34e9781c9cec6ff8bec115710273df8ded723ea8f22a37be3a627b/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/44.242.152.37/tcp/6180/noise-ik/0x614912d50b34e9781c9cec6ff8bec115710273df8ded723ea8f22a37be3a627b/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"5ee6ee045f34d87bcfac120c45aeb8e8973aee4196643341dac5e6b34df496fa"}}
2022-09-01T07:00:14.545571232Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:14.496718Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer b77026ce at /ip4/3.133.88.245/tcp/6180/noise-ik/0x3829ed5aabd623fdb814ff22a62293630f1736dfb7a5d5fddcdf8fe4c59e892c/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/3.133.88.245/tcp/6180/noise-ik/0x3829ed5aabd623fdb814ff22a62293630f1736dfb7a5d5fddcdf8fe4c59e892c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"b77026ce272a63b7261d20e5d0d9ca8cddd81b42b3432891668c43c03dbd1b73"}}
2022-09-01T07:00:16.675893058Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:16.572351Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 1e048926 at /ip4/179.48.251.12/tcp/6180/noise-ik/0xd5c8dbad5c37d224b40461bdde30d388a8e7f396e5939f80548d15a3c105b829/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.12/tcp/6180/noise-ik/0xd5c8dbad5c37d224b40461bdde30d388a8e7f396e5939f80548d15a3c105b829/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"1e04892640120d294c35cc85033634af8e93c04574771c1335d4ea6d89dbfb6e"}}
2022-09-01T07:00:18.473137667Z {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T07:00:18.320768Z","message":"[validator,Validator,050c8d86] Outbound connection failed for peer 1ce1a4ea at /ip4/35.232.235.205/tcp/6180/noise-ik/0x9adc8525186bbd01ee6ce7e8b58a9fec0b83d674868881b1afd578a5fc473f52/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/35.232.235.205/tcp/6180/noise-ik/0x9adc8525186bbd01ee6ce7e8b58a9fec0b83d674868881b1afd578a5fc473f52/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"1ce1a4ea4755a027843adc6ea9f13e158cc45edcb26be55b194b0fd278cfc041"}}

address: 0xae050e3c60e8f4e7817e0aaed4007e64a3d10b713f85f7ff28a7a7fc0d3f80e8

Logs:

aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/components/block_tree/mod.rs:246"},"thread_name":"con sensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:55.682538Z","mes sage":"Updated with a new root block","data":{"name":"speculation_cache","root_b lock_id":"1be29fc24b92345272e3f071975bddd2a6c9b0cba45d0a1bc244bdeabaeceb58"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"state_sync_dri ver","file":"state-sync/state-sync-v2/state-sync-driver/src/driver.rs:266"},"thr ead_name":"state-sync-driver","hostname":"11a4ef1d8572","timestamp":"2022-08-31T 19:38:55.684175Z","data":{"message":"Received a consensus commit notification! T otal transactions: 202, events: 0","name":"consensus_notification"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:55.970421Z","data":{"block_hash":"8a d898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e","block_parent_ha sh":"3542bd83b63cea532c191ed637d7fb262860267a4fb4f729f59e726f8270dde3","epoch":2 7,"event":"ReceiveProposal","remote_peer":"7246f46a4b1a0dfaff0687358794736f9e6c2 4faefcf72cebc7791c2c00f7810","round":8936}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:55.970491Z","message":"Local state S yncInfo[certified_round: 8935, ordered_round: 8934, timeout round: 0, commit_inf o: BlockInfo: [epoch: 27, round: 8934, id: 1be29fc2, executed_state_id: bb599c6a , version: 50973428, timestamp (us): 1661974734149092, next_epoch_state: None]], remote state SyncInfo[certified_round: 8936, ordered_round: 8935, timeout round : 0, commit_info: BlockInfo: [epoch: 27, round: 8933, id: 3a26a978, executed_sta te_id: 1e9da100, version: 50973226, timestamp (us): 1661974733758024, next_epoch _state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer" :"7246f46a4b1a0dfaff0687358794736f9e6c24faefcf72cebc7791c2c00f7810","round":8936 }}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","host name":"11a4ef1d8572","timestamp":"2022-08-31T19:38:55.973782Z","message":"Starti ng new round: NewRoundEvent: [round: 8937, reason: QCReady, timeout: 2s]","data" :{"round":8937}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:55.973801Z","data":{"epoch":27,"even t":"NewRound","reason":"QCReady","round":8937}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:55.996400Z","message":"Vote: [vote d ata: VoteData: [block id: 8ad898d7, epoch: 27, round: 8937, timestamp: 166197473 5425699,parent_block_id: 3542bd83, parent_block_round: 8936, parent_timestamp: 1 661974734999851], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, executed_state_id: 41434355, version: 0, timestamp (us): 1661974734999851, next_epoch_state: None]]]","data": {"epoch":27,"event":"Vote","remote_peer":"42fff934f0321f6235c81a2ef8baf77245fb75 78cc572acc3928c6f25b78b39e","round":8937}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.032602Z","message": "Receive ordered block BlockInfo: [epoch: 27, round: 8935, id: c00ecd56, execute d_state_id: 41434355, version: 0, timestamp (us): 1661974734597918, next_epoch_s tate: None], the queue size is 1"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.032636Z","message": "Advance execution root from None to Some(HashValue(c00ecd5638f3ffb609789d40fbad 8ac8ccf04cd793d3205db53e3b73d64687b8))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","ho stname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.033360Z","message":"exec ute_block","data":{"block_id":"c00ecd5638f3ffb609789d40fbad8ac8ccf04cd793d3205db 53e3b73d64687b8","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:56.108819Z","data":{"block_hash":"60 ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a","block_parent_ha sh":"8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e","epoch":2 7,"event":"ReceiveProposal","remote_peer":"42fff934f0321f6235c81a2ef8baf77245fb7 578cc572acc3928c6f25b78b39e","round":8937}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:56.108908Z","message":"Local state S yncInfo[certified_round: 8936, ordered_round: 8935, timeout round: 0, commit_inf o: BlockInfo: [epoch: 27, round: 8934, id: 1be29fc2, executed_state_id: bb599c6a , version: 50973428, timestamp (us): 1661974734149092, next_epoch_state: None]], remote state SyncInfo[certified_round: 8937, ordered_round: 8936, timeout round : 0, commit_info: BlockInfo: [epoch: 27, round: 8934, id: 1be29fc2, executed_sta te_id: bb599c6a, version: 50973428, timestamp (us): 1661974734149092, next_epoch _state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer" :"42fff934f0321f6235c81a2ef8baf77245fb7578cc572acc3928c6f25b78b39e","round":8937 }}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","host name":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.129745Z","message":"Starti ng new round: NewRoundEvent: [round: 8938, reason: QCReady, timeout: 2s]","data" :{"round":8938}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:56.129784Z","data":{"epoch":27,"even t":"NewRound","reason":"QCReady","round":8938}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:56.160442Z","message":"Vote: [vote d ata: VoteData: [block id: 60ee3db3, epoch: 27, round: 8938, timestamp: 166197473 5973010,parent_block_id: 8ad898d7, parent_block_round: 8937, parent_timestamp: 1 661974735425699], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: 41434355, version: 0, timestamp (us): 1661974735425699, next_epoch_state: None]]]","data": {"epoch":27,"event":"Vote","remote_peer":"0a5d60ca5c94b53c75dfc19d8d1ecc624b1056 5143ca90199c071089eb86173c","round":8938}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:346"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.304772Z","message": "Receive executed response BlockInfo: [epoch: 27, round: 8935, id: c00ecd56, exe cuted_state_id: b3f78d4c, version: 50973730, timestamp (us): 1661974734597918, n ext_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.351792Z","message": "Advance execution root from Some(HashValue(c00ecd5638f3ffb609789d40fbad8ac8ccf0 4cd793d3205db53e3b73d64687b8)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.351829Z","message": "Advance signing root from None to Some(HashValue(c00ecd5638f3ffb609789d40fbad8a c8ccf04cd793d3205db53e3b73d64687b8))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.352142Z","message": "Receive ordered block BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, execute d_state_id: 41434355, version: 0, timestamp (us): 1661974734999851, next_epoch_s tate: None], the queue size is 2"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.352181Z","message": "Advance execution root from None to Some(HashValue(3542bd83b63cea532c191ed637d7 fb262860267a4fb4f729f59e726f8270dde3))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","ho stname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.352725Z","message":"exec ute_block","data":{"block_id":"3542bd83b63cea532c191ed637d7fb262860267a4fb4f729f 59e726f8270dde3","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:393"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.353964Z","message": "Receive signing response BlockInfo: [epoch: 27, round: 8935, id: c00ecd56, exec uted_state_id: b3f78d4c, version: 50973730, timestamp (us): 1661974734597918, ne xt_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.354957Z","message": "Advance signing root from Some(HashValue(c00ecd5638f3ffb609789d40fbad8ac8ccf04c d793d3205db53e3b73d64687b8)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:346"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.495485Z","message": "Receive executed response BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, exe cuted_state_id: e72cd8ed, version: 50973830, timestamp (us): 1661974734999851, n ext_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.528168Z","message": "Advance execution root from Some(HashValue(3542bd83b63cea532c191ed637d7fb262860 267a4fb4f729f59e726f8270dde3)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.528195Z","message": "Advance signing root from None to Some(HashValue(3542bd83b63cea532c191ed637d7fb 262860267a4fb4f729f59e726f8270dde3))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:393"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.529947Z","message": "Receive signing response BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, exec uted_state_id: e72cd8ed, version: 50973830, timestamp (us): 1661974734999851, ne xt_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.530566Z","message": "Advance signing root from Some(HashValue(3542bd83b63cea532c191ed637d7fb26286026 7a4fb4f729f59e726f8270dde3)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:297"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.684938Z","message": "Advance head to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.685005Z","message": "Advance execution root from None to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.685013Z","message": "Advance signing root from None to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/block_executor.rs:232"},"thread_name":"consensus","ho stname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.685808Z","message":"comm it_block","data":{"block_id":"3542bd83b63cea532c191ed637d7fb262860267a4fb4f729f5 9e726f8270dde3","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/components/block_tree/mod.rs:246"},"thread_name":"con sensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:56.743000Z","mes sage":"Updated with a new root block","data":{"name":"speculation_cache","root_b lock_id":"3542bd83b63cea532c191ed637d7fb262860267a4fb4f729f59e726f8270dde3"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"state_sync_dri ver","file":"state-sync/state-sync-v2/state-sync-driver/src/driver.rs:266"},"thr ead_name":"state-sync-driver","hostname":"11a4ef1d8572","timestamp":"2022-08-31T 19:38:56.748777Z","data":{"message":"Received a consensus commit notification! T otal transactions: 402, events: 0","name":"consensus_notification"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:57.174733Z","data":{"block_hash":"eb c65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1","block_parent_ha sh":"60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a","epoch":2 7,"event":"ReceiveProposal","remote_peer":"0a5d60ca5c94b53c75dfc19d8d1ecc624b105 65143ca90199c071089eb86173c","round":8938}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:57.174802Z","message":"Local state S yncInfo[certified_round: 8937, ordered_round: 8936, timeout round: 0, commit_inf o: BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, executed_state_id: e72cd8ed , version: 50973830, timestamp (us): 1661974734999851, next_epoch_state: None]], remote state SyncInfo[certified_round: 8938, ordered_round: 8937, timeout round : 0, commit_info: BlockInfo: [epoch: 27, round: 8934, id: 1be29fc2, executed_sta te_id: bb599c6a, version: 50973428, timestamp (us): 1661974734149092, next_epoch _state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer" :"0a5d60ca5c94b53c75dfc19d8d1ecc624b10565143ca90199c071089eb86173c","round":8938 }}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","host name":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.178442Z","message":"Starti ng new round: NewRoundEvent: [round: 8939, reason: QCReady, timeout: 2s]","data" :{"round":8939}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:57.178469Z","data":{"epoch":27,"even t":"NewRound","reason":"QCReady","round":8939}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.178473Z","message": "Receive ordered block BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, execute d_state_id: 41434355, version: 0, timestamp (us): 1661974735425699, next_epoch_s tate: None], the queue size is 1"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus ","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.178490Z","message": "Advance execution root from None to Some(HashValue(8ad898d7fb31a826922aa0bce4bd 1e83e6dcac7b68b04deba9611c601ca31e0e))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","fil e":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","ho stname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.179596Z","message":"exec ute_block","data":{"block_id":"8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba 9611c601ca31e0e","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","fi le":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":" 11a4ef1d8572","timestamp":"2022-08-31T19:38:57.224139Z","message":"Vote: [vote d ata: VoteData: [block id: ebc65a9b, epoch: 27, round: 8939, timestamp: 166197473 6383254,parent_block_id: 60ee3db3, parent_block_round: 8938, parent_timestamp: 1 661974735973010], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 41434355, version: 0, timestamp (us): 1661974735973010, next_epoch_state: None]]]","data": {"epoch":27,"event":"Vote","remote_peer":"170c36cfb0294b4e50ea54c66a501ecd776ee9 611f0dcce314e9416626619b70","round":8939}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:346"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.377421Z","message":"Receive executed response BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: dab441a5, version: 50974264, timestamp (us): 1661974735425699, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.396599Z","message":"Advance execution root from Some(HashValue(8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.396622Z","message":"Advance signing root from None to Some(HashValue(8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:393"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.398184Z","message":"Receive signing response BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: dab441a5, version: 50974264, timestamp (us): 1661974735425699, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.398893Z","message":"Advance signing root from Some(HashValue(8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.821404Z","data":{"block_hash":"a41f41df6e5db6c4af2452ec33b4dd48a926d52fc095058af51175ec05534f9e","block_parent_hash":"ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1","epoch":27,"event":"ReceiveProposal","remote_peer":"170c36cfb0294b4e50ea54c66a501ecd776ee9611f0dcce314e9416626619b70","round":8939}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.821456Z","message":"Local state SyncInfo[certified_round: 8938, ordered_round: 8937, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, executed_state_id: e72cd8ed, version: 50973830, timestamp (us): 1661974734999851, next_epoch_state: None]], remote state SyncInfo[certified_round: 8939, ordered_round: 8938, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, executed_state_id: e72cd8ed, version: 50973830, timestamp (us): 1661974734999851, next_epoch_state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer":"170c36cfb0294b4e50ea54c66a501ecd776ee9611f0dcce314e9416626619b70","round":8939}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.824448Z","message":"Starting new round: NewRoundEvent: [round: 8940, reason: QCReady, timeout: 2s]","data":{"round":8940}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.824465Z","data":{"epoch":27,"event":"NewRound","reason":"QCReady","round":8940}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.824491Z","message":"Receive ordered block BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 41434355, version: 0, timestamp (us): 1661974735973010, next_epoch_state: None], the queue size is 2"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.824518Z","message":"Advance execution root from None to Some(HashValue(60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.824736Z","message":"execute_block","data":{"block_id":"60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.858838Z","message":"Vote: [vote data: VoteData: [block id: a41f41df, epoch: 27, round: 8940, timestamp: 1661974737395521,parent_block_id: ebc65a9b, parent_block_round: 8939, parent_timestamp: 1661974736383254], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8939, id: ebc65a9b, executed_state_id: 41434355, version: 0, timestamp (us): 1661974736383254, next_epoch_state: None]]]","data":{"epoch":27,"event":"Vote","remote_peer":"89d079d0935ed64d8a755bfe97938150bd56131336d656b31db0eacdbf03b545","round":8940}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:346"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.887701Z","message":"Receive executed response BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 2b126d60, version: 50974336, timestamp (us): 1661974735973010, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.895630Z","message":"Advance execution root from Some(HashValue(60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.895649Z","message":"Advance signing root from None to Some(HashValue(60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:393"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.897193Z","message":"Receive signing response BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 2b126d60, version: 50974336, timestamp (us): 1661974735973010, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:57.897829Z","message":"Advance signing root from Some(HashValue(60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.381051Z","data":{"block_hash":"e03eb5170d19132e2bedabf9c1a62e54bf12f7d94fdc13b66a2707fb273a64c9","block_parent_hash":"a41f41df6e5db6c4af2452ec33b4dd48a926d52fc095058af51175ec05534f9e","epoch":27,"event":"ReceiveProposal","remote_peer":"89d079d0935ed64d8a755bfe97938150bd56131336d656b31db0eacdbf03b545","round":8940}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.381103Z","message":"Local state SyncInfo[certified_round: 8939, ordered_round: 8938, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8936, id: 3542bd83, executed_state_id: e72cd8ed, version: 50973830, timestamp (us): 1661974734999851, next_epoch_state: None]], remote state SyncInfo[certified_round: 8940, ordered_round: 8939, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: dab441a5, version: 50974264, timestamp (us): 1661974735425699, next_epoch_state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer":"89d079d0935ed64d8a755bfe97938150bd56131336d656b31db0eacdbf03b545","round":8940}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.384698Z","message":"Starting new round: NewRoundEvent: [round: 8941, reason: QCReady, timeout: 2s]","data":{"round":8941}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.384742Z","message":"Receive ordered block BlockInfo: [epoch: 27, round: 8939, id: ebc65a9b, executed_state_id: 41434355, version: 0, timestamp (us): 1661974736383254, next_epoch_state: None], the queue size is 3"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.384756Z","message":"Advance execution root from None to Some(HashValue(ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.384787Z","data":{"epoch":27,"event":"NewRound","reason":"QCReady","round":8941}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.385259Z","message":"execute_block","data":{"block_id":"ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:447"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.386048Z","message":"Receive commit decision BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: dab441a5, version: 50974264, timestamp (us): 1661974735425699, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:297"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.386401Z","message":"Advance head to Some(HashValue(60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.386553Z","message":"Advance signing root from None to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/block_executor.rs:232"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.387142Z","message":"commit_block","data":{"block_id":"8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.437692Z","message":"Vote: [vote data: VoteData: [block id: e03eb517, epoch: 27, round: 8941, timestamp: 1661974738020237,parent_block_id: a41f41df, parent_block_round: 8940, parent_timestamp: 1661974737395521], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8940, id: a41f41df, executed_state_id: 41434355, version: 0, timestamp (us): 1661974737395521, next_epoch_state: None]]]","data":{"epoch":27,"event":"Vote","remote_peer":"f399f294c725bbcc8840845c4286a85324cd65e45271d88ed6d72074f25c1598","round":8941}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/components/block_tree/mod.rs:246"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.459559Z","message":"Updated with a new root block","data":{"name":"speculation_cache","root_block_id":"8ad898d7fb31a826922aa0bce4bd1e83e6dcac7b68b04deba9611c601ca31e0e"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/driver.rs:266"},"thread_name":"state-sync-driver","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.461058Z","data":{"message":"Received a consensus commit notification! Total transactions: 434, events: 0","name":"consensus_notification"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:346"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.579646Z","message":"Receive executed response BlockInfo: [epoch: 27, round: 8939, id: ebc65a9b, executed_state_id: c383fb0e, version: 50974541, timestamp (us): 1661974736383254, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.579836Z","message":"Advance execution root from Some(HashValue(ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.579848Z","message":"Advance signing root from None to Some(HashValue(ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:393"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.581539Z","message":"Receive signing response BlockInfo: [epoch: 27, round: 8939, id: ebc65a9b, executed_state_id: c383fb0e, version: 50974541, timestamp (us): 1661974736383254, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.582241Z","message":"Advance signing root from Some(HashValue(ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1)) to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:293"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.810805Z","data":{"block_hash":"5c2b013cc7adf885504f155df33ad7100889170a72ad3dd93bbf84bbdcffe938","block_parent_hash":"e03eb5170d19132e2bedabf9c1a62e54bf12f7d94fdc13b66a2707fb273a64c9","epoch":27,"event":"ReceiveProposal","remote_peer":"f399f294c725bbcc8840845c4286a85324cd65e45271d88ed6d72074f25c1598","round":8941}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:323"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.810856Z","message":"Local state SyncInfo[certified_round: 8940, ordered_round: 8939, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8937, id: 8ad898d7, executed_state_id: dab441a5, version: 50974264, timestamp (us): 1661974735425699, next_epoch_state: None]], remote state SyncInfo[certified_round: 8941, ordered_round: 8940, timeout round: 0, commit_info: BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 2b126d60, version: 50974336, timestamp (us): 1661974735973010, next_epoch_state: None]]","data":{"epoch":27,"event":"ReceiveNewCertificate","remote_peer":"f399f294c725bbcc8840845c4286a85324cd65e45271d88ed6d72074f25c1598","round":8941}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/liveness/round_state.rs:255"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.814255Z","message":"Starting new round: NewRoundEvent: [round: 8942, reason: QCReady, timeout: 2s]","data":{"round":8942}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.814271Z","message":"Receive ordered block BlockInfo: [epoch: 27, round: 8940, id: a41f41df, executed_state_id: 41434355, version: 0, timestamp (us): 1661974737395521, next_epoch_state: None], the queue size is 3"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:230"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.814287Z","data":{"epoch":27,"event":"NewRound","reason":"QCReady","round":8942}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:201"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.814290Z","message":"Advance execution root from None to Some(HashValue(a41f41df6e5db6c4af2452ec33b4dd48a926d52fc095058af51175ec05534f9e))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/block_executor.rs:183"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.815256Z","message":"execute_block","data":{"block_id":"a41f41df6e5db6c4af2452ec33b4dd48a926d52fc095058af51175ec05534f9e","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:447"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.815642Z","message":"Receive commit decision BlockInfo: [epoch: 27, round: 8938, id: 60ee3db3, executed_state_id: 2b126d60, version: 50974336, timestamp (us): 1661974735973010, next_epoch_state: None]"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:297"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.815771Z","message":"Advance head to Some(HashValue(ebc65a9b4785a7f47db131dce195da727efb40a6f862ca444d3b8eb9cba546d1))"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:229"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.815807Z","message":"Advance signing root from None to None"}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/block_executor.rs:232"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.815994Z","message":"commit_block","data":{"block_id":"60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a","name":"block_executor"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/components/block_tree/mod.rs:246"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.835500Z","message":"Updated with a new root block","data":{"name":"speculation_cache","root_block_id":"60ee3db30e68c25adbd352445960fe2901d8cf4837958c7fe6f8139953e7664a"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/driver.rs:266"},"thread_name":"state-sync-driver","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.835827Z","data":{"message":"Received a consensus commit notification! Total transactions: 72, events: 0","name":"consensus_notification"}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/round_manager.rs:547"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.865511Z","message":"Vote: [vote data: VoteData: [block id: 5c2b013c, epoch: 27, round: 8942, timestamp: 1661974738566728,parent_block_id: e03eb517, parent_block_round: 8941, parent_timestamp: 1661974738020237], author: ae050e3c, is_timeout: false, LedgerInfo: [commit_info: BlockInfo: [epoch: 27, round: 8941, id: e03eb517, executed_state_id: 41434355, version: 0, timestamp (us): 1661974738020237, next_epoch_state: None]]]","data":{"epoch":27,"event":"Vote","remote_peer":"5be2700c076c27678184fad6af04a0541fd0f2f1ed66c49cca5fe16a466807f9","round":8942}}
aptos_testnet-validator-1 | {"level":"INFO","source":{"package":"consensus","file":"consensus/src/experimental/buffer_manager.rs:297"},"thread_name":"consensus","hostname":"11a4ef1d8572","timestamp":"2022-08-31T19:38:58.884367Z","message":"Advance head to Some(HashValue(a41f41df6e5db6c4af2452ec33b4dd48a926d52fc095058af51175ec05534f9e))"}
aptos_testnet-validator-1 | {"level":"INFO","source":

wallet 0xb1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988

yesterday my sync got stuck for a few hours. I had to reinstall the validator.
after a few hours, node was successfully synchronized, but aptos_consensus_proposals_count = 0 for a long time

aptos_state_sync_version{type="applied_transaction_outputs"} 71789507
aptos_state_sync_version{type="executed_transactions"} 69702710
aptos_state_sync_version{type="synced"} 71789507
aptos_state_sync_version{type="synced_epoch"} 33

this leads to the fact that in the leaderboard the value REWARDS PERFORMANCE is also 0 (before reinstalling, it was about 80% and increased)

how to fix it??

  ```

"epoch": 23,
"failed_proposal_count": 1,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 24,
"failed_proposal_count": 1,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 25,
"failed_proposal_count": null,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 26,
"failed_proposal_count": 1,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 27,
"failed_proposal_count": null,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 28,
"failed_proposal_count": 1,
"max_potential_reward": 1598299213,
"rewards_earned": 0,
"successful_proposal_count": null,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 29,
"failed_proposal_count": null,
"max_potential_reward": 1598299213,
"rewards_earned": 1598299213,
"successful_proposal_count": 3,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100012465638064
},
{
"epoch": 30,
"failed_proposal_count": null,
"max_potential_reward": 1598324756,
"rewards_earned": 1598324755,
"successful_proposal_count": 24,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100014063937277
},
{
"epoch": 31,
"failed_proposal_count": 1,
"max_potential_reward": 1598350299,
"rewards_earned": 1465154440,
"successful_proposal_count": 11,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100015662262032
},
{
"epoch": 32,
"failed_proposal_count": 2,
"max_potential_reward": 1598373713,
"rewards_earned": 1331978094,
"successful_proposal_count": 10,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100017127416472
},
{
"epoch": 33,
"failed_proposal_count": 3,
"max_potential_reward": 1598395000,
"rewards_earned": 799197499,
"successful_proposal_count": 3,
"validator_addr": "0x8c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f",
"voting_power": 100018459394566

**address:  **0x08c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f**

aptos_consensus_proposals_count 63
aptos_state_sync_version{type="synced"} 76615017
aptos_connections{direction="inbound",network_id="Validator",peer_id="08c3f260",role_type="validator"} 88
aptos_connections{direction="outbound",network_id="Validator",peer_id="08c3f260",role_type="validator"} 132**

When my node has 99% of sync it gets stuck. Even after restart it stuck again right before it gets 100%. All updates done.
I tried to restart it deleting Data Volume, the issue still present.

REWARDS PERFORMANCE - 0 , proposals number - 0

I´m using virtual server:

10 cores CPU
RAM 60 GB
2000 GB of Disk

Ubuntu 20.04
Current commit is 4219d75c57115805f0faf0ad07c17becf6580202

Located in Germany

logs AIT3.txt

{"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:535"},"thread_name":"state-sync-driver","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:53.933762Z","data":{"message":"Committed a new transaction chunk! Transaction total: 2000, event total: 3479","name":"storage_synchronizer"}}
{"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:56.268904Z","message":"[validator,Validator,050c8d86] Dialing peer 352eb70e at /ip4/18.230.129.115/tcp/6180/noise-ik/0x56e7fe079d9c8a60f70125d76acdc49d5ec795e4ff3518eb684c37a9bc0f914e/handshake/0","data":{"network_address":"/ip4/18.230.129.115/tcp/6180/noise-ik/0x56e7fe079d9c8a60f70125d76acdc49d5ec795e4ff3518eb684c37a9bc0f914e/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"352eb70e6ae3f13fd132bb56a79ddaa8b1ce41e4f26f3a8dc058c737728dc474"}}
{"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:434"},"thread_name":"data-streaming-service","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:57.427376Z","data":{"error":"TimeoutWaitingForResponse("Rpc timed out")","event":"response_error","name":"storage_service_response","peer":"Validator:f2e80275","request_id":7005,"request_type":"get_transaction_outputs_with_proof_compressed"}}
{"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:57.427734Z","message":"[validator,Validator,050c8d86] Error making outbound rpc request with request_id 108 to f2e80275: Error: Rpc timed out\n\nStack backtrace:\n 0: network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n at ./aptos/network/src/protocols/rpc/mod.rs:477:57\n <core::future::from_generator::GenFuture as core::future::future::Future>::poll\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n at ./aptos/network/src/protocols/rpc/mod.rs:490:37\n <core::future::from_generator::GenFuture as core::future::future::Future>::poll\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n 1: <core::pin::Pin

as core::future::future::Future>::poll\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n <futures_util::stream::futures_unordered::FuturesUnordered as futures_core::stream::Stream>::poll_next\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17\n futures_util::stream::stream::StreamExt::poll_next_unpin\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9\n 2: <futures_util::stream::stream::select_next_some::SelectNextSome as core::future::future::Future>::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/select_next_some.rs:34:36\n <core::pin::Pin

as core::future::future::Future>::poll\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n futures_util::future::future::FutureExt::poll_unpin\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:562:9\n network::peer::Peer::start::{{closure}}::{{closure}}::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n 3: core::ops::function::impls::<impl core::ops::function::FnMut for &mut F>::call_mut\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:290:13\n network::peer::Peer::start::{{closure}}::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n <futures_util::future::poll_fn::PollFn as core::future::future::Future>::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/poll_fn.rs:56:9\n 4: network::peer::Peer::start::{{closure}}\n at ./aptos/network/src/peer/mod.rs:254:13\n 5: <core::future::from_generator::GenFuture as core::future::future::Future>::poll\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n tokio::runtime::task::core::CoreStage::poll::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n tokio::loom::std::unsafe_cell::UnsafeCell::with_mut\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n tokio::runtime::task::core::CoreStage::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n tokio::runtime::task::harness::poll_future::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n <core::panic::unwind_safe::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n std::panicking::try::do_call\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n std::panicking::try\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n std::panic::catch_unwind\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n tokio::runtime::task::harness::poll_future\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n 6: tokio::runtime::task::harness::Harness<T,S>::poll_inner\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n tokio::runtime::task::harness::Harness<T,S>::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n 7: tokio::runtime::task::raw::RawTask::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n tokio::runtime::task::LocalNotified::run\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:376:9\n tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:435:13\n tokio::coop::with_budget::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:102:9\n std::thread::local::LocalKey::try_with\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:445:16\n std::thread::local::LocalKey::with\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:421:9\n 8: tokio::coop::with_budget\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:95:5\n tokio::coop::budget\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:72:5\n tokio::runtime::thread_pool::worker::Context::run_task\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:434:9\n 9: tokio::runtime::thread_pool::worker::Context::run\n 10: tokio::runtime::thread_pool::worker::run::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:386:17\n tokio::macros::scoped_tls::ScopedKey::set\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/macros/scoped_tls.rs:61:9\n 11: tokio::runtime::thread_pool::worker::run\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:383:5\n 12: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:362:45\n <tokio::runtime::blocking::task::BlockingTask as core::future::future::Future>::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/task.rs:42:21\n 13: tokio::runtime::task::core::CoreStage::poll::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n tokio::loom::std::unsafe_cell::UnsafeCell::with_mut\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n tokio::runtime::task::core::CoreStage::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n tokio::runtime::task::harness::poll_future::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n <core::panic::unwind_safe::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n std::panicking::try::do_call\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n std::panicking::try\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n std::panic::catch_unwind\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n tokio::runtime::task::harness::poll_future\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n tokio::runtime::task::harness::Harness<T,S>::poll_inner\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n tokio::runtime::task::harness::Harness<T,S>::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n 14: tokio::runtime::task::raw::RawTask::poll\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n tokio::runtime::task::UnownedTask::run\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:413:9\n tokio::runtime::blocking::pool::Task::run\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:91:9\n tokio::runtime::blocking::pool::Inner::run\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:325:17\n 15: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}\n at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:300:13\n std::sys_common::backtrace::rust_begin_short_backtrace\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18\n 16: std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:505:17\n <core::panic::unwind_safe::AssertUnwindSafe as core::ops::function::FnOnce<()>>::call_once\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n std::panicking::try::do_call\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n std::panicking::try\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n std::panic::catch_unwind\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n std::thread::Builder::spawn_unchecked::{{closure}}\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:504:30\n core::ops::function::FnOnce::call_once{{vtable.shim}}\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5\n 17: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n <alloc::boxed::Box<F,A> as core::ops::function::FnOnce>::call_once\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n std::sys::unix::thread::Thread::new::thread_start\n at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys/unix/thread.rs:108:17\n 18: start_thread\n at ./build/glibc-fWwxX8/glibc-2.28/nptl/pthread_create.c:486:8\n 19: clone\n at ./build/glibc-fWwxX8/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e"},"remote_peer":"f2e802753bbaa2b3e0861180dfd2dd3fe8dc9bb0d49e0094376f6d565bab9884"}}
{"level":"ERROR","source":{"package":"data_streaming_service","file":"state-sync/state-sync-v2/data-streaming-service/src/data_stream.rs:413"},"thread_name":"data-streaming-service","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:57.525176Z","data":{"error":{"AptosDataClientError":"Timed out waiting for a response: Rpc timed out"},"event":"error","message":"Encountered a data client error!","name":"received_data_response","stream_id":2}}
{"level":"ERROR","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/driver.rs:602"},"thread_name":"state-sync-driver","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:58.126733Z","data":{"error":{"DataStreamNotificationTimeout":"5s"},"message":"Error found when checking the bootstrapper progress!","name":"driver"}}
{"level":"INFO","source":{"package":"executor","file":"execution/executor/src/chunk_executor.rs:304"},"thread_name":"state-sync-driver","hostname":"77c2a2749b30","timestamp":"2022-09-01T00:10:59.077722Z","message":"Applied transaction output chunk!","data":{"first_version_in_request":46257610,"local_synced_version":46257609,"name":"chunk_executor","num_txns_in_request":2000}}

I'm trying to make my node synchronized on 10 cores/60 RAM/1.6TB SSD server (Contabo XL VPS, Germany).

It worked fine until 4219d75c57115805f0faf0ad07c17becf6580202 update. Now it can't achieve 100% sync. Working via docker.

What i've tried:

1. Checked all files with keys (all is correct).
2. I had no problems joining validator set.
3. Deleted secure-data.json from docker container.
4. Increased stake lockup period.
5. Upgraded my server (it was on 4/8/200 before).
6. Restarted docker, deleting all volumes (sync took over 10 hours).

After that node was working for few epochs, but then stuck again. Now it can't get into sync again.

Owner address: 0x81bd0427d12a5240dede6df96b99f1b70fabdd8acc05a79dad4c71910564bcdd

Logs:
lenkaziakova_logs.txt

• My VPS for this AIT3 has currently the following specs:
  8 vCPUs
  32 GB RAM
  240 GB NVMe SSD
  1Gbps connection
  20 TB Transfer available
  Ubuntu 20.04 (LTS) x64
  Located in Helsinki/Finland

• I'm running from Docker (image tag - testnet_4219d75c57115805f0faf0ad07c17becf6580202)

• My validator can no longer sync even after restarting it, as a consequence it is also no longer voting or proposing.

• I am sharing my attached log.

stdout.log

My validator have 64GB RAM and only 2GB RAM is free

Hello! I've found my node 168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764 in list, but my validator-identity is correct (account address is my owner).
Successful transactions:

"epoch": 37,
        "failed_proposal_count": 2,
        "max_potential_reward": 1598351194,
        "rewards_earned": 1278680955,
        "successful_proposal_count": 8,
        "validator_addr": "0x168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764",
        "voting_power": 100015718278584

Metrics:

aptos_connections{direction="inbound",network_id="Validator",peer_id="168625c7",role_type="validator"} 149
aptos_connections{direction="inbound",network_id="vfn",peer_id="00000000",role_type="validator"} 1
aptos_connections{direction="outbound",network_id="Validator",peer_id="168625c7",role_type="validator"} 70
aptos_connections{direction="outbound",network_id="vfn",peer_id="00000000",role_type="validator"} 0
# HELP aptos_consensus_proposals_count Count of the block proposals sent by this validator since last restart (both primary and secondary)
# TYPE aptos_consensus_proposals_count counter
aptos_consensus_proposals_count 1
# HELP aptos_data_client_highest_advertised_data Gauge related to the highest advertised data
# TYPE aptos_data_client_highest_advertised_data gauge
aptos_data_client_highest_advertised_data{data_type="ledger_infos"} 106199825
aptos_data_client_highest_advertised_data{data_type="states"} 106199825
aptos_data_client_highest_advertised_data{data_type="transaction_outputs"} 106199825
aptos_data_client_highest_advertised_data{data_type="transactions"} 106199825
aptos_state_sync_version{type="synced"} 106199825

Hardware:
16CPU/32GB/SSD/600Gbit

Logs:

ait3-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.176493Z","message":"[validator,Validator,168625c7] Outbound connection failed for peer 1dd17527 at /ip4/167.235.26.108/tcp/6180/noise-ik/0xa6bf1ee023a3a51c554752e9db07d554a69c36e7938c5404d4e33917f9004a5c/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/167.235.26.108/tcp/6180/noise-ik/0xa6bf1ee023a3a51c554752e9db07d554a69c36e7938c5404d4e33917f9004a5c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"1dd17527c01616b252c9801366c158fbc0ff67b5ee76436e3cec04399d8de5d0"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.176546Z","message":"[validator,Validator,168625c7] Failed to connect to peer: 1dd17527 at address: /ip4/167.235.26.108/tcp/6180/noise-ik/0xa6bf1ee023a3a51c554752e9db07d554a69c36e7938c5404d4e33917f9004a5c/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/167.235.26.108/tcp/6180/noise-ik/0xa6bf1ee023a3a51c554752e9db07d554a69c36e7938c5404d4e33917f9004a5c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"1dd17527c01616b252c9801366c158fbc0ff67b5ee76436e3cec04399d8de5d0"}}
ait3-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.263696Z","message":"[validator,Validator,168625c7] Outbound connection failed for peer f1d4c93a at /ip4/18.116.40.89/tcp/6180/noise-ik/0xb08371db4a44697c5e5dbe6a9f9e4c1512aaa5a8c9ee62288ff942ec4f017f4a/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/18.116.40.89/tcp/6180/noise-ik/0xb08371db4a44697c5e5dbe6a9f9e4c1512aaa5a8c9ee62288ff942ec4f017f4a/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"f1d4c93aa7e20e194e339a7d23c63b09a5501595870910c666213afea6ea1239"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.263749Z","message":"[validator,Validator,168625c7] Failed to connect to peer: f1d4c93a at address: /ip4/18.116.40.89/tcp/6180/noise-ik/0xb08371db4a44697c5e5dbe6a9f9e4c1512aaa5a8c9ee62288ff942ec4f017f4a/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/18.116.40.89/tcp/6180/noise-ik/0xb08371db4a44697c5e5dbe6a9f9e4c1512aaa5a8c9ee62288ff942ec4f017f4a/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"f1d4c93aa7e20e194e339a7d23c63b09a5501595870910c666213afea6ea1239"}}
ait3-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.311615Z","message":"[validator,Validator,168625c7] Outbound connection failed for peer c3f2c9dc at /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.311666Z","message":"[validator,Validator,168625c7] Failed to connect to peer: c3f2c9dc at address: /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:805"},"thread_name":"aptos-data-client","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.361290Z","data":{"event":"aggregate_summary","message":"Global data summary: AdvertisedData { Highest epoch ending ledger info, epoch: Some(45). Highest synced ledger info, epoch: Some(46), version: Some(106392861). Lowest transaction version: Some(0), Lowest transaction output version: Some(0), Lowest states version: Some(572224) }, OptimalChunkSizes { epoch_chunk_size: 100, state_chunk_size: 2000, transaction_chunk_size: 2000, transaction_output_chunk_size: 2000 }","name":"peer_states"}}
ait3-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.426358Z","message":"[validator,Validator,168625c7] Outbound connection failed for peer d16b8e15 at /ip4/66.94.104.49/tcp/6180/noise-ik/0x6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/66.94.104.49/tcp/6180/noise-ik/0x6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"d16b8e15be631f0d22199f8b91b7338862fef9ee0a3af419477bd148a26cd2ff"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.426408Z","message":"[validator,Validator,168625c7] Failed to connect to peer: d16b8e15 at address: /ip4/66.94.104.49/tcp/6180/noise-ik/0x6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/66.94.104.49/tcp/6180/noise-ik/0x6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"d16b8e15be631f0d22199f8b91b7338862fef9ee0a3af419477bd148a26cd2ff"}}
ait3-validator-1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.474680Z","message":"[validator,Validator,168625c7] Outbound connection failed for peer 3a9bcf67 at /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.474749Z","message":"[validator,Validator,168625c7] Failed to connect to peer: 3a9bcf67 at address: /ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.134.37.36/tcp/6180/noise-ik/0xa5c30c777336df4716887460a63ff7fa441129f5ec6071c57d84ce61e15df625/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"3a9bcf67be83271b05a2b3587e92b9a86834f7b281b35b636f9a2637398a9bf0"}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:593"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:47.687118Z","message":"[validator,Validator,168625c7] Received response for expired request_id 566 from 4c2cccd2. Discarding.","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"4c2cccd23aeec74fdd6a4309d77c15a382d1afad35fa0cd3f258be16be414789","request_id":566}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:593"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:48.553903Z","message":"[validator,Validator,168625c7] Received response for expired request_id 783 from 873cc8a2. Discarding.","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"873cc8a29a8ad6afe1d13ffbaa512a91be22bb1b513ac4bd0b1077200d6be96f","request_id":783}}
ait3-validator-1 | {"level":"INFO","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:593"},"thread_name":"network-Validator","hostname":"d1b432822f75","timestamp":"2022-09-02T08:07:49.174188Z","message":"[validator,Validator,168625c7] Received response for expired request_id 568 from e35d670f. Discarding.","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764"},"remote_peer":"e35d670fa6481a8e490b33c0d33a2030831b996b5638e4ffc81c2864943bd6b4","request_id":568}}

Owner address: 0x46774d7bda97925b0f7dc7eecc927fe4f2f17739f087243043694d2bea153cf5

My validator fails to send proposals after update.

Current commit is 4219d75c57115805f0faf0ad07c17becf6580202

log is here.

log.txt

After updating to testnet_88f936bf40a56768e2dac420e953a0307281c1ba version I often don't have proposals, sometimes failed proposals (before network pauses and updates I had only one failed proposal for the whole time and received successful proposals in every epoch)

Logs: https://drive.google.com/file/d/1ZeoJ5CL2dQ7cj0tRO9qOGyvEKhqBjCxd/view?usp=sharing

Owner address: 0xea1272b8a1bc5a1fb9a1c77c9dc96e1a02a691d1d6073a1dc06d51d770b3f2e2
Build type: docker
OS: Ubuntu 20.04

Attaching grafana dashboards and latest epochs results:

Please let me know if you need any additional information.

after writing this command
curl 127.0.0.1:9101/metrics 2> /dev/null | grep "aptos_connections{.*\"Validator\".*}"

I see

aptos_connections{direction="inbound",network_id="Validator",peer_id="c3035340",role_type="validator"} 164
aptos_connections{direction="outbound",network_id="Validator",peer_id="c3035340",role_type="validator"} 0

checking logs with

docker logs -f --tail 100

find this

Outbound connection failed for peer afea3644 at /ip4/5.188.4.254/tcp/6180/noise-ik/0x2948ff5016e8abc49d453f94fb12adf2c77671e85c68665d56a2b75a54bc2e09/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of `file","network_address":"/ip4/5.188.4.254/tcp/6180/noise-ik/0x2948ff5016e8abc49d453f94fb12adf2c77671e85c68665d56a2b75a54bc2e09/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"c303534096a966dc42b45dad6ee63657056568f4b2f6836a78922f7a1b84baac"},"remote_peer":"afea364425825255938bbfdb1704f06ec56d5c914833208b96fc47b94aff1671"}}

Synchronized. Number keep increasing proposals within 12 hours, but not see in leaderboard. REWARDS PERFORMANCE go lower. Using testnet_88f936bf40a56768e2dac420e953a0307281c1ba (last update)

OWNER ADDRESS
0x9eaa6c899597ca4d691cf453a1b0b17b534ed8fa91eac78a9e9bc0f457ee8cba

root@aptos3:~/aptos-core# git pull
remote: Enumerating objects: 6887, done.
remote: Counting objects: 100% (3331/3331), done.
remote: Compressing objects: 100% (298/298), done.
remote: Total 6887 (delta 3085), reused 3210 (delta 3024), pack-reused 3556
Receiving objects: 100% (6887/6887), 4.64 MiB | 1.43 MiB/s, done.
Resolving deltas: 100% (4800/4800), completed with 969 local objects.
From https://github.com/aptos-labs/aptos-core

• 479faa84e7...b6d0241a7d testnet -> origin/testnet (forced update)

• [new branch] 0822alden-more-pruned-version-check -> origin/0822alden-more-pruned-version-check
• [new branch] 0825alden-long-and-hard -> origin/0825alden-long-and-hard
• [new branch] 0826alden-remove-mirai -> origin/0826alden-remove-mirai
• [new branch] 0831alden-panic-deque -> origin/0831alden-panic-deque
• [new branch] Fei94043-patch-5 -> origin/Fei94043-patch-5
• [new branch] Fei94043-patch-6 -> origin/Fei94043-patch-6
• [new branch] Fei94043-patch-7 -> origin/Fei94043-patch-7
• [new branch] Fei94043-patch-8 -> origin/Fei94043-patch-8
• [new branch] alin/codeowner -> origin/alin/codeowner
• [new branch] alin/move-bulletproofs -> origin/alin/move-bulletproofs
• [new branch] backlog_tweak -> origin/backlog_tweak
• [new branch] brian/epoch-repro -> origin/brian/epoch-repro
• [new branch] buy -> origin/buy
• [new branch] c-audit -> origin/c-audit
• [new branch] ci-config -> origin/ci-config
• [new branch] commit_blocks_ext -> origin/commit_blocks_ext
• [new branch] community-ui-nav-jh -> origin/community-ui-nav-jh
• [new branch] consensus_backpressure -> origin/consensus_backpressure
• [new branch] delegations -> origin/delegations
• [new branch] delete_unused_api -> origin/delete_unused_api

• 509f09e786...8399cd1c7b devnet -> origin/devnet (forced update)

• [new branch] error_msg_fix -> origin/error_msg_fix
• [new branch] ev -> origin/ev
• [new branch] exp_branch_9575518 -> origin/exp_branch_9575518
• [new branch] explain-token-abi -> origin/explain-token-abi
• [new branch] fi2 -> origin/fi2
• [new branch] fix-genesis -> origin/fix-genesis
• [new branch] fix-timeout-issue -> origin/fix-timeout-issue
• [new branch] igor/6k_mix_forge -> origin/igor/6k_mix_forge
• [new branch] igor/add_consensus_stress_test -> origin/igor/add_consensus_stress_test

• 3ffccdee03...c57bd787be igor/add_epoch_change_to_forge -> origin/igor/add_epoch_change_to_forge (forced update)

• [new branch] igor/enable_wait_for_catchup_2 -> origin/igor/enable_wait_for_catchup_2
• [new branch] igor/smoke_ulimit -> origin/igor/smoke_ulimit
• [new branch] igor/test_10k_tps -> origin/igor/test_10k_tps
• [new branch] igor/test_8k_tps -> origin/igor/test_8k_tps
• [new branch] igor/txn-emitter-followup -> origin/igor/txn-emitter-followup
• [new branch] indexer_to_block -> origin/indexer_to_block
• [new branch] jpark/boogie-install-doc -> origin/jpark/boogie-install-doc
• [new branch] jpark/prove-stake -> origin/jpark/prove-stake
• [new branch] jpark/set-genesis-end -> origin/jpark/set-genesis-end
• [new branch] key-rotation -> origin/key-rotation
• [new branch] local_swarm_mint -> origin/local_swarm_mint
  5392bf8008..a8634d39a3 main -> origin/main
• [new branch] messthingsup -> origin/messthingsup
• [new branch] move_state_value_writer -> origin/move_state_value_writer
• [new branch] multied25519 -> origin/multied25519
• [new branch] observability -> origin/observability
• [new branch] owner-token -> origin/owner-token
• [new branch] package-publish-py-sdk -> origin/package-publish-py-sdk
  17dc740c90..4f5d85fd2f quorum-store -> origin/quorum-store
• [new branch] quorum-store-cache -> origin/quorum-store-cache
• [new branch] raj/reflow-docs -> origin/raj/reflow-docs
• [new branch] raj/rest-api-docs -> origin/raj/rest-api-docs
• [new branch] revert-3451-conversions -> origin/revert-3451-conversions
  1570509e74..52dc7eee07 rosetta-stable -> origin/rosetta-stable
• [new branch] rustielin/docker-failpoints -> origin/rustielin/docker-failpoints
• [new branch] rustielin/forge-no-sticky -> origin/rustielin/forge-no-sticky
• [new branch] rustielin/grafana-sync -> origin/rustielin/grafana-sync
• [new branch] sdk-key-rotation -> origin/sdk-key-rotation
• [new branch] sherrynet_with_fast_indexer -> origin/sherrynet_with_fast_indexer
• [new branch] signer_capability -> origin/signer_capability
• [new branch] ss_perf_test -> origin/ss_perf_test
• [new branch] sync-grafana-dashboards -> origin/sync-grafana-dashboards
• [new branch] test_igor/account_creation_loadtest -> origin/test_igor/account_creation_loadtest
• [new branch] test_timeout -> origin/test_timeout
• [new branch] testnet3 -> origin/testnet3
• [new branch] testnet4 -> origin/testnet4
• [new branch] testnet_igor/txn_emitter_const_tps -> origin/testnet_igor/txn_emitter_const_tps
• [new branch] upgrade_old_indexer -> origin/upgrade_old_indexer
• [new branch] yield_threads -> origin/yield_threads

• 668353904c...3e44b6f0ec zach/it3 -> origin/zach/it3 (forced update)

• [new branch] zekun/consensus -> origin/zekun/consensus
• [new tag] aptos-cli-v0.3.1a -> aptos-cli-v0.3.1a
• [new tag] aptos-cli-v0.3.2 -> aptos-cli-v0.3.2
• [new tag] cli-v0.3.1 -> cli-v0.3.1
• [new tag] wallet-v0.1.7 -> wallet-v0.1.7
• [new tag] wallet-v0.1.8 -> wallet-v0.1.8

*** Please tell me who you are.

Run

git config --global user.email "[email protected]"
git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'root@aptos3.(none)')
root@aptos3:~/aptos-core# git log
commit 479faa84e78580aad913781054acd917c0aeb77a (HEAD -> testnet, tag: aptos-cli-v0.3.1)
Author: Sherry Xiao [email protected]
Date: Fri Aug 19 14:25:33 2022 -0700

[cli] bump version

commit ceed7d658761f11e39d905749e7f8f11b262b225
Author: Rustie Lin [email protected]
Date: Fri Aug 19 13:45:24 2022 -0700

[helm][aptos-node/fullnode] bump down cpu and mem requirements (#3203)

commit 7b00476a83ea157eeece39b42284c5798e10b013
Author: Max Kaplan [email protected]
Date: Fri Aug 19 13:34:05 2022 -0700

[CP] Update AIT3 dates + links (#3202)

commit 47e4581378950f21e3c6a67ae527fa049fe51c15
Author: Sherry Xiao [email protected]
Date: Fri Aug 19 13:15:00 2022 -0700

[docs] update command for framework (#3201)

commit 8cb049dbe9ebd7fb7d8614ad0f4921c9e5011ddb (tag: aptos-framework-v0.3.0)
Author: Kent White [email protected]
:...skipping...
commit 479faa84e78580aad913781054acd917c0aeb77a (HEAD -> testnet, tag: aptos-cli-v0.3.1)
Author: Sherry Xiao [email protected]
Date: Fri Aug 19 14:25:33 2022 -0700

[cli] bump version

commit ceed7d658761f11e39d905749e7f8f11b262b225
Author: Rustie Lin [email protected]
Date: Fri Aug 19 13:45:24 2022 -0700

[helm][aptos-node/fullnode] bump down cpu and mem requirements (#3203)

commit 7b00476a83ea157eeece39b42284c5798e10b013
Author: Max Kaplan [email protected]
Date: Fri Aug 19 13:34:05 2022 -0700

[CP] Update AIT3 dates + links (#3202)

commit 47e4581378950f21e3c6a67ae527fa049fe51c15
Author: Sherry Xiao [email protected]
Date: Fri Aug 19 13:15:00 2022 -0700

[docs] update command for framework (#3201)

commit 8cb049dbe9ebd7fb7d8614ad0f4921c9e5011ddb (tag: aptos-framework-v0.3.0)
Author: Kent White [email protected]
Date: Fri Aug 19 13:01:23 2022 -0700

[docs] Move wallet dapp transaction back to json (#3177)

commit 753547edf8648d293ea26fe63001555318f7220f
Author: Zach Denton [email protected]
Date: Fri Aug 19 15:40:40 2022 -0400

:

Excuse me my request #7 got closed for some reason.

But since epoch 18 and up until now i get absolutely 0 rewards per epoch... What can be the cause please?

I made all the necessary updates, my node is connected to Aptos node but i dont generate any proposals or consensus rounds.

Please help me!

address: 0x848750a4781af9d181cbbf9ecd0ee8eb04c62c048f207a4ef23864334e39b01e

During some epoch transitions I see a proposal get dropped every now and again.
I do not see anything on the resource side with my nodes, both VFN and V to indicate pressure of any kind. What I do see is unusual correlation to TCP connections both outbound and inbound:

It seems to be the same problems others are experiencing when they stall completely but my node is recovering within a minute. It seems like some kind of deadlock is occurring when epoch is in transition and the node re-attempts a batch of connections to new/previously unavailable peers.

After I updated my validator image to "testnet_4219d75c57115805f0faf0ad07c17becf6580202", this happened. But I am not sure that update caused the issue. (I changed my validator image back to older one, and still getting the same issue).

Log file :
verse2-log.txt

When trying to cast a vote on a proposal 0 we receive an failed transaction error:

Transaction 0xcb8ccdbe1ec0a89ce5e56b78eb4513e8d73688f8fbbfa303b79207af200ca2ac failed with "Execution failed in 0x1::stake::get_delegated_voter at code offset 1"

https://explorer.devnet.aptos.dev/txn/0xcb8ccdbe1ec0a89ce5e56b78eb4513e8d73688f8fbbfa303b79207af200ca2ac

owner address: 0x57be1bdaf544178f3e754dd23f84c74a95785c694066f6c1d48762b8c888cbee
voter address: 0x25a5ffaae84b3bb526ead2f28ae1f3e322dbe77f0013aad1d7be6e2a3e67614e

My node is experiencing quite a bit of these:

2022-09-01T05:34:48.666747Z [consensus] ERROR consensus/src/epoch_manager.rs:651 {"error":"[EpochManager] Verify event\n\nCaused by:\n 0: Failed to verify Commit Vote\n 1: Author is unknown","remote_peer":"87a572ffb4a0c22b32d41b91d1df342da0d4477b234faa394f0328bed498a1be","security-event":"consensus_invalid_message","unverified_event":{"CommitVote":{"author":"5465c144c24e524029ec67232f79a33bee8198b4e2a5760227acf6557f472e67","ledger_info":{"commit_info":{"epoch":32,"round":7581,"id":"4438f683821ef3b61bb60539845acf4e2d3f49e27a4f42541b194ab2ee6345c2","executed_state_id":"76a13bc5cb504de67ad7ddc8f3ad511e8b7d3d0eb4916eddbadf850acd618ab7","version":68955296,"timestamp_usecs":1662010486134120,"next_epoch_state":null},"consensus_data_hash":"3cc4d559d2e22cc4459ad33786f4ca1b3638c7409a7f53a4abe471f2c024e775"},"signature":"0xb6a39320f54a54b123c9796c1a3cd263c1c56831ac1c3ebf8a751e64680839b8c6b644aa818865eb3331ea41025c7f64197e188564669e2385fb1e7036559cee32b776329240e5b055ada8c1950790c6b69a7da7b077f6832a7175616553870d"}}} 2022-09-01T05:34:48.666764Z [consensus] ERROR consensus/src/epoch_manager.rs:789 {"epoch":32,"error":"[EpochManager] Verify event\n\nCaused by:\n 0: Failed to verify Commit Vote\n 1: Author is unknown","kind":"VerifyError"} 2022-09-01T05:34:48.777802Z [consensus] ERROR consensus/src/epoch_manager.rs:651 {"error":"[EpochManager] Verify event\n\nCaused by:\n 0: Failed to verify Commit Vote\n 1: Author is unknown","remote_peer":"87a572ffb4a0c22b32d41b91d1df342da0d4477b234faa394f0328bed498a1be","security-event":"consensus_invalid_message","unverified_event":{"CommitVote":{"author":"5465c144c24e524029ec67232f79a33bee8198b4e2a5760227acf6557f472e67","ledger_info":{"commit_info":{"epoch":32,"round":7583,"id":"168de9eda98e272ce707d3d731e7e13205547aa09a1c28ebeb3bfde31bc08125","executed_state_id":"d6149c5bbe2448e9be72b8b31ca01bfb23da02dfdce65fbd2c2f97b6a571fd58","version":68955800,"timestamp_usecs":1662010487209252,"next_epoch_state":null},"consensus_data_hash":"647d51ec7fe845e35bba1019d1a685c0671c31290e18efa610aee56febb81057"},"signature":"0xaf037a5fbd1e9ef210361b219c4c72d739023e65586675b742f2e67234e143cb89fbc2f20f6199d6c87cd0a781447abc0d7e6c8b8d4369f9bdcca2b03f969816420fa3c6e9e763f02d858f3d4b5cbf9860ea03ed69f0ec2161f3b92f0466c5da"}}} 2022-09-01T05:34:48.777819Z [consensus] ERROR consensus/src/epoch_manager.rs:789 {"epoch":32,"error":"[EpochManager] Verify event\n\nCaused by:\n 0: Failed to verify Commit Vote\n 1: Author is unknown","kind":"VerifyError"}

VPS located in Brazil

8 vCPU / 16GB RAM / 320GB SSD / 10GBit/s / Ubuntu 20.04 / docker (testnet_4219d75c57115805f0faf0ad07c17becf6580202)
Owner address and petra's wallet address: 0xd2f842ded238f6ba4629650863dc3ebbd506f857c6e1050a4ac923fc60e2e4a1

1st of all - my node is generating abnormal outbound traffic (more than 1500 GB per 6 hours):

My node has been fully synced:

My validator-identity.yaml is correct (it contains owner address as it has been suggested by Sherry):

I'm in validator set list as you may see:

My node liveness is 100%.

My node is connected to many aptos peers as well:

Here is my log snapshot file:
https://docs.google.com/document/d/1Ij9lANcoonjEbWoKfz3LrxZavoioEFNFEaI3KkvMTeM/edit?usp=sharing

upd
now my node spoiled 3TB of outbound traffic

Node just annihilate tons of outbound traffic and proposals counter works partly, then freezes - reboot needed to make counter work (until next freeze because of out of sync state). That's why my reward rate is too weak.

@sherry-x Any ideas?

validator_1 | {"level":"INFO","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:805"},"thread_name":"aptos-data-client","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:01.104623Z","data":{"event":"aggregate_summary","message":"Global data summary: AdvertisedData { Highest epoch ending ledger info, epoch: Some(29). Highest synced ledger info, epoch: Some(30), version: Some(60240623). Lowest transaction version: Some(0), Lowest transaction output version: Some(0), Lowest states version: Some(534412) }, OptimalChunkSizes { epoch_chunk_size: 100, state_chunk_size: 2000, transaction_chunk_size: 2000, transaction_output_chunk_size: 2000 }","name":"peer_states"}}
validator_1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/chunk_executor.rs:304"},"thread_name":"state-sync-driver","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:01.398688Z","message":"Applied transaction output chunk!","data":{"first_version_in_request":55579298,"local_synced_version":55579297,"name":"chunk_executor","num_txns_in_request":2000}}
validator_1 | {"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:444"},"thread_name":"state-sync-driver","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:01.398908Z","data":{"message":"Applied a new transaction output chunk! Transaction total: 2000.","name":"storage_synchronizer"}}
validator_1 | {"level":"WARN","source":{"package":"network","file":"network/src/peer_manager/transport.rs:309"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:02.632061Z","message":"[validator,Validator,03c02943] Inbound connection from /ip4/66.94.104.49/tcp/35970 failed to upgrade after 0.001 secs: noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376","data":{"error":"noise server: client 6dd21ff8: client connecting with unauthenticated peer id: 6dd21ff8d10578768466ccdc97b42e3acc557d016034d767171843090cde5376","network_address":"/ip4/66.94.104.49/tcp/35970","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"}}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:359"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.360596Z","message":"[validator,Validator,03c02943] New connection established: [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/154.26.128.150/tcp/45224/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":237,"connection_origin":"inbound","network_address":"/ip4/154.26.128.150/tcp/45224/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:637"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.360645Z","message":"[validator,Validator,03c02943] Closing existing connection with Peer 28dde00b to mitigate simultaneous dial","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:350"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.360804Z","message":"[validator,Validator,03c02943] Closing connection to peer: 28dde00b","data":{"connection_id":236,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:370"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.360836Z","message":"[validator,Validator,03c02943] Connection [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/5.161.137.197/tcp/38520/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator] closed due to Requested","data":{"connection_id":236,"connection_origin":"inbound","disconnection_reason":"Requested","network_address":"/ip4/5.161.137.197/tcp/38520/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:384"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.360920Z","message":"[validator,Validator,03c02943] Closed connection to peer: 28dde00b","data":{"connection_id":236,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"WARN","source":{"package":"network","file":"network/src/peer_manager/transport.rs:309"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:04.394705Z","message":"[validator,Validator,03c02943] Inbound connection from /ip4/18.230.78.80/tcp/48572 failed to upgrade after 0.001 secs: noise server: client 351a3e91: client connecting with unauthenticated peer id: 351a3e91731204baa596e094366152a77b8d4454065fdc7eff1c6879bddf48a6","data":{"error":"noise server: client 351a3e91: client connecting with unauthenticated peer id: 351a3e91731204baa596e094366152a77b8d4454065fdc7eff1c6879bddf48a6","network_address":"/ip4/18.230.78.80/tcp/48572","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"}}}
validator_1 | {"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:535"},"thread_name":"state-sync-driver","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.231277Z","data":{"message":"Committed a new transaction chunk! Transaction total: 2000, event total: 3579","name":"storage_synchronizer"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:359"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.286654Z","message":"[validator,Validator,03c02943] New connection established: [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/5.161.137.197/tcp/54658/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":238,"connection_origin":"inbound","network_address":"/ip4/5.161.137.197/tcp/54658/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:637"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.286721Z","message":"[validator,Validator,03c02943] Closing existing connection with Peer 28dde00b to mitigate simultaneous dial","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:370"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.286854Z","message":"[validator,Validator,03c02943] Connection [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/154.26.128.150/tcp/45224/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator] closed due to Requested","data":{"connection_id":237,"connection_origin":"inbound","disconnection_reason":"Requested","network_address":"/ip4/154.26.128.150/tcp/45224/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:350"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.286860Z","message":"[validator,Validator,03c02943] Closing connection to peer: 28dde00b","data":{"connection_id":237,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:384"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:07.286971Z","message":"[validator,Validator,03c02943] Closed connection to peer: 28dde00b","data":{"connection_id":237,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
validator_1 | {"level":"INFO","source":{"package":"executor","file":"execution/executor/src/chunk_executor.rs:304"},"thread_name":"state-sync-driver","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:09.185454Z","message":"Applied transaction output chunk!","data":{"first_version_in_request":55581298,"local_synced_version":55581297,"name":"chunk_executor","num_txns_in_request":2000}}
validator_1 | {"level":"INFO","source":{"package":"state_sync_driver","file":"state-sync/state-sync-v2/state-sync-driver/src/storage_synchronizer.rs:444"},"thread_name":"state-sync-driver","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:09.185751Z","data":{"message":"Applied a new transaction output chunk! Transaction total: 2000.","name":"storage_synchronizer"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:238"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.018059Z","message":"Current connected peers","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"peers":"[(b45b9b8600c15bc8b1c0eb857ffa5211c9cf106db8f12a7ca2555c3a5966f0b9, /ip4/3.34.176.23/tcp/55884/noise-ik/0x2d75d5782e69b851802d492ba7eefd46ec42ef31cee60ca5495eda6dd2b9eb31/handshake/0, inbound), (170c36cfb0294b4e50ea54c66a501ecd776ee9611f0dcce314e9416626619b70, /ip4/34.71.104.5/tcp/1094/noise-ik/0x7e862ca6529eb9fcdcb341d761530fee096394bba5b9621d1926561ba1b6d92b/handshake/0, inbound), (c725187215cf9ba7a54f3dfca3472d986eec830065eab4d667dc271483ac5728, /ip4/13.244.203.209/tcp/34814/noise-ik/0xae938e5574af17fd7bd9adb5e2a2f73c7c9982853760c0eb476147555414977a/handshake/0, inbound), (86ed41e7b82e5f22b54b5b2a9c331cf92edbd5b5419de0cb188aeb2e4bc2eba0, /ip4/52.62.250.204/tcp/33930/noise-ik/0xf118e98de16e90aef2ca3eaf5ecbfc1c701ab6d0cdb8765ce120569eeaafd965/handshake/0, inbound), (07d429d25c23497040431c6ddfde65dba6dbb8702876fb1f060f00bd435420a9, /ip4/34.64.155.34/tcp/1169/noise-ik/0x876ab35b691fe2251dde8d27d33f2b71123da8ae3dd7534a445e4a3ced800772/handshake/0, inbound), (50c7f72106ed1991c454a876a9bac3f0bb7dce470c1719e94cde43be5b55f12d, /ip4/34.131.25.44/tcp/34834/noise-ik/0x4710a28049efde9b1f2631822086684ac0c2b290d56a2a733754ef02b1524e78/handshake/0, inbound), (fcfa1efb940e23936915c2a6814ca557dbf3c6972d4ff0154d8e6aacda905c64, /ip4/65.21.133.41/tcp/49820/noise-ik/0x7c2399a2208d886b1cada4086d34c38c56016dd85e828fe021fb7e81c0a1743c/handshake/0, inbound), (23f422eb4212100c47cdc28c242cab052b65ace49d1429c1197f4a1b153b178e, /ip4/13.213.60.209/tcp/36308/noise-ik/0x574397cd405ed717f3884f85dd7867481aad75b3b57e9c1fe0f93ab223d2e613/handshake/0, inbound), (b12cf85a0a28c36b5703ff9aa5fcc995a3c64f8869f68f61ffbc17b1b6814d25, /ip4/34.88.117.19/tcp/46572/noise-ik/0x8a6552abe0beea67214d6e531ed54627a9113d09506241d3edcf3c266d654108/handshake/0, inbound), (b1bc346da7756a51dae71ddbd8d74228604598fa6cc35a52fb78e86ca3915e26, /ip4/65.108.247.142/tcp/47736/noise-ik/0x784c747e4b8d6e8bdf0523948a795b2f745bff81d81c848c08808aefbf6e2550/handshake/0, inbound), (873cc8a29a8ad6afe1d13ffbaa512a91be22bb1b513ac4bd0b1077200d6be96f, /ip4/45.76.112.117/tcp/59368/noise-ik/0xf871eaf06dda4809661d4ffed410374a03ff6bbd696f7bc739110c3107623f59/handshake/0, inbound), (e35d670fa6481a8e490b33c0d33a2030831b996b5638e4ffc81c2864943bd6b4, /ip4/5.188.133.205/tcp/37186/noise-ik/0x34344a0ffe8b50238ef6631debdb5a02e46dfd6827cfbfc60330015aa0fac853/handshake/0, inbound), (35a254e3c5a42534b8e7f08e5ce3cd9fa28ba20e6b1b9f228dc38b4275ade609, /ip4/92.100.157.81/tcp/43876/noise-ik/0x3de95af8190ebeff2013adbba2b4d85e3c324cbbf8088b827d29f9560a85f867/handshake/0, inbound), (0f33a11bc2cc943b451e17e4d74dfc7fcc3af2bf5a6ba6dfefffc4f7c7e95d04, /ip4/14.160.32.134/tcp/57640/noise-ik/0x5680c60e4d0912ad3b070cf0988393a0d032f91b64cfcc3bb51a7192ef5eb70a/handshake/0, inbound), (309da7c8f94a4454364810dfc0f6f317079cc382d872e73a9e32fbf4fbf70ef1, /ip4/34.233.209.153/tcp/4163/noise-ik/0x6789ddc9a971a5fb5cf4b90b1a38fa31f4aa5904ce8e21e61c53379411cb5569/handshake/0, inbound), (528a460b6482d633fbfc9d9e9a6b5a2e47f2749c35a50d2416124674cb46bc1f, /ip4/135.181.252.139/tcp/51896/noise-ik/0xc7d31b8d608e59113dff7eee67344d41c540eea13807c955a0bdb509b9dd585c/handshake/0, inbound), (c2371a6ccfdc035723f58b67df18fdcc28b1579522670680732c741063eb55ff, /ip4/142.132.131.15/tcp/35718/noise-ik/0xa2bc90355843c042715f7bb74d2e0ccb6685a89836edfacbc3c1810e1f33c74a/handshake/0, inbound), (21d0a3f424d63546ede086a3fb950a5d6bccd431b991374e1f037914e2e45bfe, /ip4/34.173.15.232/tcp/32799/noise-ik/0xed45926cd25b944e6252124e5c8ca74f79bca475ad28ebbd782698a023e0e45c/handshake/0, inbound), (ea1272b8a1bc5a1fb9a1c77c9dc96e1a02a691d1d6073a1dc06d51d770b3f2e2, /ip4/135.181.221.183/tcp/40446/noise-ik/0xb5967ab631ad33d89f9b43ec1a77eb04b22a140fa0bfae99a19f701b7d5c2d0b/handshake/0, inbound), (f7c045f174d8527bd300eb5e9e84fb6db43a9e80190f7b9597db095fa0995658, /ip4/54.77.138.64/tcp/42205/noise-ik/0xf7ccfe6d8502eb9729a5d92b4e5ce5f6c5ce0b8e51e34eb231668800db5e4f22/handshake/0, inbound), (c11fffec54e78cb992ff9d34e46ea601d5a84c35985ae516f125bd4c0c5449e3, /ip4/50.21.189.252/tcp/42318/noise-ik/0x80407ef18c8a9bdc03a09a1d766204147aa8b7b66e37b3efb192513bcf147509/handshake/0, inbound), (b10dc0857cfb72773f71f11cb2a466eba3e1947d1b8cd1bf2b3124e936fb5468, /ip4/185.209.164.105/tcp/34288/noise-ik/0x1cfffdd639287067bee0f79b0fa0b583ad8a088c34a7d848eb67a5cbbfdd790a/handshake/0, inbound), (8e4d6379237ec2576f9b71513ee566e524793310027ae6271977d481d6d07bf4, /ip4/18.162.195.229/tcp/51552/noise-ik/0xb03b2f8c6457c09adfece2a766372b84ed1f12b89ee44c7aec4f55bf0756ee42/handshake/0, inbound), (00fa666ddf3c506305d2f6226d19f2df4c25c09f8537cecd4fd8a5b381366d3b, /ip4/194.99.78.13/tcp/37796/noise-ik/0xdf48faef1646f2241a7f783b27ed86bb632c7140412faae6a3c5ad7986f05a43/handshake/0, inbound), (eb15bc1d36806e5079d8699c404e3aba86cb136751be943969d6373570c88b7e, /ip4/142.44.226.224/tcp/39800/noise-ik/0x506064e642b017a297675314a33ad8316cc581ee40dde39c8bdcd04f9053a853/handshake/0, inbound), (3406fda6b62a60cae162193b9dcb59c871b9cf8e150a792ca2d433bf8a040e49, /ip4/49.12.186.156/tcp/44658/noise-ik/0x70abcd0d2359f05c13bea3e8275760b12c7418fc043d1c01f08172a2b6d4730f/handshake/0, inbound), (e586f8a8c30b8cc949b9642c5c8d41548dc4558385656e46cd5b5be45a2c9d2a, /ip4/35.201.16.25/tcp/42738/noise-ik/0x4a2b5018185d0b65d123049553dc3e4170a9c6b27256e263d2a78dcadfadee5d/handshake/0, inbound), (cce4914c224ee34157924f910bb08ca30d5e66b56ca094d50dba2bd2adb8b80b, /ip4/23.88.5.243/tcp/37820/noise-ik/0xc5d9f0aaab4f43d0b2bd2a0499de1d5b0a7a1860fe5d7fbc9858c767f5af4275/handshake/0, inbound), (ed8f98dd4c607ea196fdd0edb9aafa68e7fe31f9eb736f93e9dc744954d960e5, /ip4/92.101.121.75/tcp/52946/noise-ik/0xd12bb15376cb42a244d082f8b904b72833d787fd24d743d2cb8b2cdc351fba6a/handshake/0, inbound), (d19590c3a471f55ac53b0fe7304206be49eb6dfcb9df93ca205007aba6a2378d, /ip4/46.101.142.248/tcp/50954/noise-ik/0xe3fe73572d76390fe4baff708d5c7546f972aff825e5377a536804488dc93372/handshake/0, inbound), (1b0867a303b9c2c975b3c874866c06fd53e5ee0e538c301b66ffcecf08e63d47, /ip4/47.254.41.217/tcp/52606/noise-ik/0x383b5dd36124185707243442d0331ca23718e6e64eeab2d5828b4b37184db439/handshake/0, inbound), (4c2cccd23aeec74fdd6a4309d77c15a382d1afad35fa0cd3f258be16be414789, /ip4/18.141.11.56/tcp/58078/noise-ik/0x4d5a4d7c8586dd4181b954dd77f3b6ba7d91aae45e5c00c9a5a340c7467bc604/handshake/0, inbound), (1a86526cab3a14ce5d3ea14d55fa313c9b284ab1543df434b9ea67f90f4ff682, /ip4/13.246.152.6/tcp/44156/noise-ik/0xfeae97146d929e5f4cd34372df785f5e9beb383709468ba5b6668d3f02887a20/handshake/0, inbound), (9a27424bc1d21dbecb89318781224a8deca73fdbcb13a7f15005ecb266e45f5e, /ip4/216.238.72.99/tcp/37202/noise-ik/0xdfa6fbdfe5e71b533050e0bf321a51e05ecd925c2e0220f45cfea4e15162f30d/handshake/0, inbound), (6ebbe079ff81335cb9b1ab2c667a15c9994b60001768b6a2236b19b4c273b71d, /ip4/154.53.61.217/tcp/40856/noise-ik/0x66e85f953202b77f72320496c71e2014643d574a007e5bddd7d75595f5bbf554/handshake/0, inbound), (46774d7bda97925b0f7dc7eecc927fe4f2f17739f087243043694d2bea153cf5, /ip4/159.69.24.237/tcp/33212/noise-ik/0x03b4d1488422ba8372375c81a30428b62a21e2a289070a3e3b737e5f5c8d1449/handshake/0, inbound), (7b689638b3a5473309f49325f87284ffca126e126f80495ad6c185e43eecb7d3, /ip4/145.40.87.115/tcp/36740/noise-ik/0x792c7a163ed9787b0340461cd9968695e914ddc9522d8f95d78f9300f85ae656/handshake/0, inbound), (1ce1a4ea4755a027843adc6ea9f13e158cc45edcb26be55b194b0fd278cfc041, /ip4/194.233.94.132/tcp/46570/noise-ik/0x9adc8525186bbd01ee6ce7e8b58a9fec0b83d674868881b1afd578a5fc473f52/handshake/0, inbound), (168625c7aca6358fe94af73c52d65615c009683f372b5b0e7068d5f306aea764, /ip4/195.54.41.122/tcp/59224/noise-ik/0xb8f3fb95a727a99e66bf24bf9a05a526d330656b55c13d93ac53c543f9848f68/handshake/0, inbound), (0a5d60ca5c94b53c75dfc19d8d1ecc624b10565143ca90199c071089eb86173c, /ip4/90.188.227.243/tcp/43472/noise-ik/0xbd8994563f25937fc9d8cec3a1583ea582ece0b8b9511f1efdf60a6f13d5062f/handshake/0, inbound), (5ee6ee045f34d87bcfac120c45aeb8e8973aee4196643341dac5e6b34df496fa, /ip4/44.235.99.102/tcp/55304/noise-ik/0x614912d50b34e9781c9cec6ff8bec115710273df8ded723ea8f22a37be3a627b/handshake/0, inbound), (454ae62855c24140f39398de478b49ac53dd072edea2b54f07d272c9008ddc12, /ip4/85.10.207.109/tcp/45050/noise-ik/0xdae7be89696e714c932483aee07b570edace3361c996871c8f116b81a1086f39/handshake/0, inbound), (0a0df6771891d5d1171a3c808aeb5914487944e82863017e500f290a377368bc, /ip4/44.196.202.54/tcp/36154/noise-ik/0x52fe2e6bcf7cfb598ff76e59fea51c90747f9a2925289dd9a6a061580211056f/handshake/0, inbound), (802f9bbd78574afe79289edf5adcce044ab23535dcd8e79289abf2cb492a3c57, /ip4/51.159.141.5/tcp/48615/noise-ik/0xfa01ddaad94935025a1fc57b494504fc8b38fbd836e1240b530bb4827e83a868/handshake/0, inbound), (be2ac53a4dba0325cf3d65eca93c68e5239c6fc5b4ca013e6ec7089235c43775, /ip4/139.99.62.189/tcp/60528/noise-ik/0x117295a9a205a59823c7598aeeb1025a0088ef555c9a6facbaa8d0853139d607/handshake/0, inbound), (0851fbcc396e403f8f910870f05d4307b275f73f96cfaaae2c06ca0b93aa77b5, /ip4/95.217.113.104/tcp/54254/noise-ik/0xb1251f6b9f8c7d25ffae475297da56ede376d192e82983e668134b1525abd151/handshake/0, inbound), (3683d546ef9af9652f789965ac4470f2a74d614e691f0c396d7bf5ad8c5ad00c, /ip4/47.88.16.18/tcp/33932/noise-ik/0x36d7561cea4a5a910775c003c0bb268a1767b02dee193fbe22b25e4137750a24/handshake/0, inbound), (22cbd4218363ab29c03b98b129f19c7c5188668514bffa99186d8689123af3bc, /ip4/34.147.52.40/tcp/37120/noise-ik/0x656f8fc7ffba079617dccee690c02f7f6032287bce4f76850aad8173d404932c/handshake/0, inbound), (b10a392979c9de9e2fb696dbb01c33d65036349e46f5ac51e1e4266c3b8847e0, /ip4/13.246.149.59/tcp/49664/noise-ik/0x61433943e63e22b8f5f8269edd8c4912a78d4427e52e60ce4ff6ece4f213f170/handshake/0, inbound), (699b1fe83796492513a17f276607a4d3dbc250ac26e9d5f455ea790776b6337e, /ip4/34.130.213.144/tcp/40712/noise-ik/0xb2783a7127cfa367378b3f5c1e4c314ca16c67b660b8225e7aff8fe2f247871d/handshake/0, inbound), (c54fc020dbf12a2f04c184e62ffd53f64a1c82f9d2bd6f4e074bed4f512148c4, /ip4/65.109.53.60/tcp/50288/noise-ik/0x103724e39cb70d3827200853c38721a57de14e2495306cfc35a8c50361c49a01/handshake/0, inbound), (848750a4781af9d181cbbf9ecd0ee8eb04c62c048f207a4ef23864334e39b01e, /ip4/75.119.152.83/tcp/39072/noise-ik/0x66b03977e4c98bce89a5e5554fdd0a79c777750e08a2d2ae9d6408358fd46d12/handshake/0, inbound), (093cc8991c3092c9a9a6262d233cfad2550abdc6f5a58422e30675ad08ccfb08, /ip4/194.99.78.7/tcp/41674/noise-ik/0xf4679ce73e34378e76587018f0cd536f070042288b6ec734cd65580c1118961d/handshake/0, inbound), (442a4f67e45a025612221d7710d3ab7bb8918d3d4bc888303baf7e6709588a1b, /ip4/54.228.5.11/tcp/25126/noise-ik/0x4777b5a38707bf80410080b0106dcff61328f4b370d54ff9e69cfff6daf21b7c/handshake/0, inbound), (bd2d5cb253a3ae97ee84039b34ec753c3bca038a41ab953b9f6df4ac38bf7d35, /ip4/52.37.113.96/tcp/26787/noise-ik/0xfc6c8debd9956c60cc794046fabd5055dde4a274ae4b982025686fafd56b0a38/handshake/0, inbound), (d801e1b2c79395df00ff26f63f60e2f11d541d18d6755831ec1a6537fd3ad6d3, /ip4/95.188.77.87/tcp/45638/noise-ik/0xcc98cf612f641136ff744bcfd0a3228219fa815e9269556b6d992c23236b462f/handshake/0, inbound), (dc0253a45c8e56c91425f1415b3227fc016e207eedb1536cebf50c9e7baec92d, /ip4/95.111.241.161/tcp/48382/noise-ik/0xd03367cb035b4137f973726a1537eb22d2c22e320f5842c2cdad6a00a0c0ac25/handshake/0, inbound), (84699c4425469b5610f8c30d66026a57b9f7d7442cdcac7b2bf11875d80b05c2, /ip4/35.86.47.196/tcp/47734/noise-ik/0x0983ed2b584484d7b8211d623ed17c2726a062ff1c7dadbbe1d821e704404831/handshake/0, inbound), (97a0efea1c4e3a9aeb9ac4a9cfa9025bb602b01d8fdaa14c9b1ab03c7be05cb8, /ip4/46.4.91.238/tcp/17482/noise-ik/0x31209c9c8f920fe34fa9a549c2949b261cbbf658e7914a826e84553221311522/handshake/0, inbound), (bb356acd63be6e072284d210b7e8e2eeae911ad512f1bc24631dcbc2c22a4a98, /ip4/65.109.27.185/tcp/60388/noise-ik/0x8b5315ad4f9efccbc1c78cbb6795d4e06569b1edb4ca08d46a960f3cd2dfef4d/handshake/0, inbound), (8a147943e71c0c9d2d167492905d5e27e6af4ee4788badde9299120abc231b7b, /ip4/138.219.42.19/tcp/47900/noise-ik/0xb9b77c73282533e56a3578fb562e38a2af452a9b357f6c1121b2f9f70d8b6d70/handshake/0, inbound), (365e00da83dd53c167be1c08437daf01146a71bd8dc859df037c6bcd27ca37fb, /ip4/115.73.213.74/tcp/6180/noise-ik/0xa82e9a75c9867ee8e561b464e37391f37585b60cf61905f7343012c677eb3811/handshake/0, outbound), (b0cc635492330102e2e0592033045f0b14381a9cbd8f43f21223fc1fa3b58ac6, /ip4/34.152.49.63/tcp/36010/noise-ik/0x156126b104d08913caa1860d398f346f3ce388a6464d9407f85df6598f3c122b/handshake/0, inbound), (a4e0e382397c09a69408b088f021a475d1e9d55c3f708660102c837440e2ef1b, /ip4/3.222.205.97/tcp/38012/noise-ik/0xd9e944cdc59c3609334305820a0418b0895b7aaf883b722f7452844f29737f3b/handshake/0, inbound), (c654a30871b504309c94fac9a164b2273b8be5d71149ed79b304ba8d89fd5c88, /ip4/95.217.215.139/tcp/56004/noise-ik/0x2982ea2047e86f59512dea9bc35b85d5308521a79f687d60ee168bf0b8ba9f0d/handshake/0, inbound), (4449959032d51d85aae24c37ad6a9c5a27499b7d7b67dfd5327e61eb2bc3051c, /ip4/162.55.243.225/tcp/58360/noise-ik/0xe528106c7a33568e502bbd1fa6ab6e319c726ae64b0ac2e31e5652c614a24664/handshake/0, inbound), (96b148b5ba586d3b0406450e793ddd416fe6f7bdd21ee8d4f0b1ff245303aa4a, /ip4/18.143.20.193/tcp/42270/noise-ik/0xd4ba4a67b0fd41e83ca81ef0e620992470bc6f7acfb6fdc892f373e7d95f9b15/handshake/0, inbound), (42fff934f0321f6235c81a2ef8baf77245fb7578cc572acc3928c6f25b78b39e, /ip4/107.23.190.98/tcp/5265/noise-ik/0xfced3d2d21cddb2c3c92930edd6e9849dc7297a82fb9fccc1bb8e1adce2d4072/handshake/0, inbound), (bec5d849323f0e3b633bdd47283ada03ca085f455e6bbca056fe91de941db4ef, /ip4/110.53.221.139/tcp/53864/noise-ik/0xe00efe1aaf304dc0d05aae9558472a59721221ce86f05aa8ee87ee0c2d5bca6d/handshake/0, inbound), (d62ae400c7622e8469c2ee52b1e0af5e5c11f6a72560bc4434694eeeb3c1ffd4, /ip4/52.78.35.143/tcp/1430/noise-ik/0x18b1b28f23a9504eb15ce186edb87e0554726988054a16bfdc912994d970995e/handshake/0, inbound), (d15c01e90855d4e89a8497c74f5915656138817ba29c56cfca85f8b5f5b7b07b, /ip4/77.94.99.53/tcp/36918/noise-ik/0x5e904959ddf7e6454cdfc1ad42753c9c58e47812cdf2ac348c22d667636ba129/handshake/0, inbound), (0289311ada457c33abfaaabff61177c935a3705ac7ac624a43e767615cad2548, /ip4/15.184.159.144/tcp/59136/noise-ik/0x43d7eed0086757d38a63aab7ef96b66128f1d8edb53c6b6410d168c1a2378a2d/handshake/0, inbound), (de102b1247b8b89979b5f58dc29da624d01b0381fd725908464e02a2d3bcec94, /ip4/70.34.213.245/tcp/50006/noise-ik/0x7cfe50ee15bad3751f885c30faa46bd6601dd8aec4d02c48b01e57732285647f/handshake/0, inbound), (750a7e5997bcb7cc87917880524d91c7a59847801b6fe0a8d2c632c0a10c9441, /ip4/109.107.188.22/tcp/57148/noise-ik/0x8910bc66c029eb2b9b44e112265d4f9c80a03f95dd8a74da36561259c43f927c/handshake/0, inbound), (b1a242d87bdc7a948db36f9c25dc8e92fcc2c66c7f090ad369d6362d70024988, /dns/aptos-ait3.uarnet.org/tcp/6180/noise-ik/0xaaa99ec567143dcde3ac726641dd760921bf11ca2c9dd961be191fadc6c94c7b/handshake/0, outbound), (9991b734f39f2f22d8bc071db5ec2d42cd074fa4fe9e9f19a12873fd7cca34df, /ip4/34.159.182.2/tcp/56416/noise-ik/0xf2a28e60096ff06485f66f4625f2b031f7f832fa05e1bd516f40128314c5605b/handshake/0, inbound), (49bbd1c4b5313d97a9218d97d48020392d3991e978886802d51d36bb747f7bf1, /ip4/194.5.152.243/tcp/37956/noise-ik/0xd5bb3846c585a9e183c1ab3fdc2f480a92075bf34f88d27bf8eb0e8821987826/handshake/0, inbound), (b14350297a3d2e2caf1a887213b4aebf6b120fa2ae87e9ddd1d34d9ad7beb44c, /ip4/5.161.148.94/tcp/48780/noise-ik/0x8d3e06177d6ac00a4c6489b70521ec210f8b0bb477f61963a099c161945a3421/handshake/0, inbound), (e0aff63df5c4b9ce3d7e415e2c0b8aa2f036f0a0d730ab31546bf45a2569d925, /ip4/13.49.202.145/tcp/54070/noise-ik/0x70a9c09fa79faabda99997ffaf8cdf91d8a0d43f060659eeeab0cdf14bdca757/handshake/0, inbound), (ae576d308450419f8ff71e72c550da533720271c7c3693b3a3bb4596a7475942, /ip4/34.131.105.170/tcp/42746/noise-ik/0x865168659ec068fcc111b97436084ddd9c0cf36d0ed3bc1cac947545bf3a530c/handshake/0, inbound), (6165c3ef6ac4c0d8ef7496a0cacd3de25384bc7909130482da3e7369bee2da24, /ip4/185.245.182.214/tcp/38742/noise-ik/0xc47c9aa12cfcbd42591d05cc17cb2101ef7b026b0a803b98d3058f8ee21e8678/handshake/0, inbound), (064ca224990b2bec38cc2db62fd079dadf36b9f03aa98582d5d7d75f98850abc, /ip4/138.68.243.58/tcp/49296/noise-ik/0xd9f189f13691e159c8d40bccc3f4e33d73f1389153f07415f1bdbca1b00e0e55/handshake/0, inbound), (f326fd3007939a18e96d9cbb68da732e06a160759ae12b1fe3d4bc0ef7066a81, /ip4/35.247.236.199/tcp/1180/noise-ik/0x14843ab9d62e1f4a2728bf98033960112fd3baf6c1ffde944349991cfaa5b046/handshake/0, inbound), (17e6eeec824c4ebe71f276aeb2115b63defcd8b5898663074d02c6db6494991b, /ip4/65.109.18.34/tcp/34800/noise-ik/0x7ed1792d893e90356d3e0a603c28bc2d0f35721c37ac99a6bc2417f564eb602c/handshake/0, inbound), (84428d86d9e5655e2951c8b57ba5e796d858e342ef037807610d73ca6ef6c409, /ip4/3.136.101.233/tcp/41998/noise-ik/0x88b21a0a439937995b8c19ad0543444d6a0e7064daa962a3e82cc46d7028a327/handshake/0, inbound), (8485b0e4779190c2e0cefc899ac853cd3d878930e66587df8d867c0c941267a6, /ip4/65.108.136.196/tcp/36884/noise-ik/0x47b0877b4d38c768b58deebc17f3fca23d199a870b01649b22942d553b23933a/handshake/0, inbound), (ed06553d095aec80bbe77c3d86004b2bfa3f14e85969fe769fe5b7928af9db42, /ip4/200.58.100.107/tcp/40148/noise-ik/0x0586ccc0c4b30f8f83c52f3faa041dd93442ebe35bbd62aecaec913c7e4fd620/handshake/0, inbound), (b6eb384b229e17db7b1f4f6fa5a692b91cf8d371e1e97e442ae7b60a0f4308d6, /ip4/216.238.66.193/tcp/39802/noise-ik/0x25c4e9a7ea7f0279649c05288ac72f7c85f285886434922517e6d44278a9f22f/handshake/0, inbound), (84081add80123b1df0da71af7108f06d5b3b1c2910915778a92f16b1318c4180, /ip4/20.55.61.82/tcp/58182/noise-ik/0xe51fea074597616cace08d77eb65307c4d7acd031a8a82ec1e3000f434e99f66/handshake/0, inbound), (ff79fc4bf0b39c4b8a9810b57d311527700639b661198f4d8db77d7c6d64f0c9, /ip4/95.217.87.117/tcp/47478/noise-ik/0x56a83d0455db2acf897c4b2cece26e76d88a99aa94359ac9d45f5a4cdbb0f661/handshake/0, inbound), (0d8fee8284ef09fe6fb272d357e31f142a8d2cef2c089aa618a6e6de76f53703, /ip4/154.53.63.176/tcp/43082/noise-ik/0x31252a074b7d794325984ac1466e2648d263c50aa745fc912177cd9a6813ff34/handshake/0, inbound), (f988e9f477963de81075449affe13bb81beddb91d7bb5ab29071ce1c5f0bc400, /ip4/149.129.236.247/tcp/59968/noise-ik/0x7cf960f70702e43ef92106b7c326721978005921c202b7682868738c55e9c550/handshake/0, inbound), (8aa4d82016b1a0356421c22cda47990d8ca5ffc0f5a012754817db871d34ae90, /ip4/216.238.81.121/tcp/46082/noise-ik/0xce47a503a93faf88c52902e255192ce418b6909741ae06bd1c43b807abaca27c/handshake/0, inbound), (2ec5fe60d0e96c4791826635be764ad9e24db9918642aff078244dcb03b24203, /ip4/99.79.96.31/tcp/23807/noise-ik/0x171f702c00513e2cd0f86ed10b89dcbd0b0303f8dab6007c7778eaf2c8c66d1f/handshake/0, inbound), (050c8d865560ff370970f4652d1c08c376ed85d68647a3b34863004d84b6388e, /ip4/149.129.223.249/tcp/48270/noise-ik/0x696666ef43661b7c4f69a837e38eeab07e98f577cbda3220a5a8efb931be7d61/handshake/0, inbound), (f399f294c725bbcc8840845c4286a85324cd65e45271d88ed6d72074f25c1598, /ip4/204.236.155.102/tcp/48554/noise-ik/0x625647490c68539675237d0c52526a003e6da5f6590622aa7341620695613875/handshake/0, inbound), (cfd596602a7d09bb54b847f1933a93ce1b3552b604d4c960c0e67a9e3d6d9532, /ip4/216.238.111.101/tcp/45810/noise-ik/0x7b0f39d628ff4ff7b84d7414b1222311f7f19f7ff199a9788f2040de177ead65/handshake/0, inbound), (e36800ba3761489f69de5ed4f0d3023d352209fe62c6fd67b3498e0914e45bb7, /ip4/208.72.154.99/tcp/6180/noise-ik/0x9c978a791d83ad66946ea5463880dfa457b31981b3ba8ee6f562a917b185cc79/handshake/0, outbound), (5c8af906ab13573a0a1a3c34f3308c0afb9e72f1b6e5d7845c7f57ba2f02c7ca, /dns/aptos.mindheartsoul.org/tcp/6180/noise-ik/0x759ccc6d87cbec8adbc58961cbe9b817de6252602e1a50a8a6485a434e059a45/handshake/0, outbound), (c4a5af5fa6c1ae769157675c8fbe55c49e2cbd7ce89a8dec5f09a6ced957568d, /ip4/99.80.230.109/tcp/56374/noise-ik/0x11c26a9b9756fb33e255edc6e4034a2b5e9caf0f6dabc58d7be3242754bef808/handshake/0, inbound), (95ce20e79294e79fca729d40db823144c677a27eec7a53c5cd021dd28602c6d7, /ip4/194.5.152.240/tcp/40644/noise-ik/0xcc5840503727a68e1495829238be0b439cdf6652a8552e111078b49d96e92820/handshake/0, inbound), (19105a85b8d455e51a1b0f917dbb2464d47c47a325c20e5aaa64916299e06aa4, /ip4/51.159.149.68/tcp/37244/noise-ik/0xb9ee72e5c7ad36879780e9590131acd924b6c68e4895b6c05df81c65d3904102/handshake/0, inbound), (a7f779852a2739af1d9d7abdf661827dd5edb4b9e3db716c30b76105676f68f1, /ip4/45.77.49.142/tcp/50622/noise-ik/0xd115a026be834530c2be047e6ddcedfd488fdc42324ec08e9c9029e5caac7b1c/handshake/0, inbound), (81bd0427d12a5240dede6df96b99f1b70fabdd8acc05a79dad4c71910564bcdd, /ip4/167.86.87.218/tcp/37148/noise-ik/0x9a59c8b7c5a374944fea36f1772173ee8ece257daa515864f18f1f15623aeb31/handshake/0, inbound), (28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186, /ip4/5.161.137.197/tcp/54658/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0, inbound), (6e86e31f81c541b4f7aaafb1170db28ba1d64fbde5ed453637c51377f7de518e, /ip4/3.112.171.175/tcp/50230/noise-ik/0xead8c09e3c9063121a5d5f2202c219891a12c4130a8454f5b03c6617ec3f5608/handshake/0, inbound), (9ab325358eafa5b89b016647e08bc6ecc7b4f070a13d0c167809c279a6a5b07d, /ip4/34.159.116.142/tcp/53312/noise-ik/0xfcf9bbc70bc64563699da6fa6f68412f7500ab368caf65427e48ee02b949972c/handshake/0, inbound), (39ac041fa2f9f85bb615d12dcd628e4b06d2b7f03fe7c52af5e2b250d27c252a, /ip4/3.218.22.136/tcp/27497/noise-ik/0x8022e58a0b38a69a62bec1f358b8d6fde64059d2ccab827f266efde11cbf2670/handshake/0, inbound), (9db9b05383bf12758faa1f81a08f4fc5ffa632266be516b48d5496d8625e352a, /ip4/158.255.76.170/tcp/50044/noise-ik/0xe5c042679121c48d5af4ab86702b5463237aff0b715da76a2ff68aaf9771494c/handshake/0, inbound), (f9e184c68cfa53ea4ba0d7354c5ecc512c0d2fa32a13c479c5560b293b7f6a35, /ip4/35.203.84.116/tcp/34752/noise-ik/0x2263c82b106ddb1401c61856c6db2f30c08f6b9e558b647a38ed549c37525c60/handshake/0, inbound), (a2445e86428742f04b3ac622343c44fd3321f352ce8bc42399605b32cc190a7e, /ip4/142.132.140.115/tcp/40758/noise-ik/0xcf118d099f7b5ac011d71def58e062c9e0d5fca12d77368735c4a47dc45b0c4a/handshake/0, inbound), (bec1cf784c7b93744687fe7899f7aeafe6fc9a7fc9c758f3f933ec9cf1668f70, /ip4/207.148.8.191/tcp/55722/noise-ik/0x15e27a9f9ed9de2167a464375d070c6dba3b9ba8557604253dde1fd8620e186a/handshake/0, inbound), (a3fdaacd44c39f1d548aaac3fb6a25336c37fc01bec7233505d50a77fddc84ae, /ip4/75.119.154.35/tcp/35332/noise-ik/0xd5c64a7ee6cda5e1f9e3555ea33f1ef872ab2d1b26b97a76483461d08581e730/handshake/0, inbound), (01a95d1f40c04eecf33d1ca9ddf2e077cb3cdc8f39132251dd26dd1560d719d8, /ip4/94.130.164.210/tcp/50780/noise-ik/0x88d9bd8004e921d5031a6ab41c49634cf593694315324693e1267b09cd848673/handshake/0, inbound), (000c5b2258a85d4ea8a313f09ff68165cb87e6e1a917f245a94949f0895fc290, /ip4/54.158.198.7/tcp/11982/noise-ik/0x8ae018ac8b0eafdc657e251cba6f4c3cb783afa70c4ba96ae2e41cc542e2aa58/handshake/0, inbound), (c3bb0a1e3406f82dae0ac03161b6816bd27bf5a861ce122e8c64e42373f1a124, /ip4/20.168.17.105/tcp/53964/noise-ik/0x0e23bf16636eccb3df7a8dafa4f7c04d6c8d07882835f031554d6ed981923c67/handshake/0, inbound), (7ec006180b2a66f14c5b48f366a96c4e75496a7a26d663e506922524cc8b0d1e, /ip4/208.72.155.244/tcp/46672/noise-ik/0xda34cec981d6265bc7ee9b7b023ed496c6fe168f11dad972ae3f76a87e9b7047/handshake/0, inbound), (bfeac9b2da698dee2d9b882322d20dc7508bb6cf2df4165c8938ffd748e11337, /ip4/65.109.21.207/tcp/54388/noise-ik/0xda0fdf6d263496b8b7851db9d2db9140d990f51f573dc81cb73bb8263ab24628/handshake/0, inbound), (935719a60ad09f4a6b74baf20133760c720b895a3c4a59d2cb3258d8b5b055a1, /ip4/177.153.20.38/tcp/55682/noise-ik/0x005184e24bc6aee7cfd5df5e435290fc242b70c807740c5bff4d314f15720e1d/handshake/0, inbound), (dc10d8a940d909cd3c26b8d2c0378d6aab4165b5d98a1f5bdc592aeaa6cdca5f, /ip4/65.108.134.120/tcp/42626/noise-ik/0x6f4c473108bc1683f3f5e6bf9e6977b8123671a4443927d5e99ca39125813e0b/handshake/0, inbound), (a6665a2c9c2ed477121ecea1047b803b1e0daa96748ee58aa66f3762dbeb2fac, /ip4/80.248.22.14/tcp/40642/noise-ik/0x1047c1a1d1f6ce3a82b26abc0f2f50bf339477675d4afa152cd36f9a85b28665/handshake/0, inbound), (479107df9ac8bb68a12900e6df3b928a1a6fcf99fbf64ac6f0c1b6b37a2c7b3a, /ip4/43.128.211.119/tcp/37448/noise-ik/0xd5ceafe4047b6328260c8458d6a218f03c721dd14f7a9bab87914e20e0c0c96a/handshake/0, inbound), (6503030aaf4e9be90bf06a172e4980a520064f839288c89b39f21d180d973c6c, /ip4/104.243.38.172/tcp/47096/noise-ik/0x7cbd1bab80b2c8cbde817b683ff0c5962eba5c9789e2f75ed481b5b63824d566/handshake/0, inbound), (916926a34055b6402d115dbd5f6b8eeebad4ba70d775e0e6da864f3f919b9100, /ip4/102.37.122.247/tcp/43642/noise-ik/0x941d1971de30b5c14a0ac4992ebd57c4b3a5870338e969d26c7b39d26cb9cd2b/handshake/0, inbound), (44194bedcb5bfac13f003eddbc6567e27ffbc5c2cb98090294b90419c522c0e7, /ip4/65.108.125.55/tcp/60038/noise-ik/0x8bce29074fa2358137bfa3d83e86c02c66220b9706f594e722283c34510e7367/handshake/0, inbound), (7246f46a4b1a0dfaff0687358794736f9e6c24faefcf72cebc7791c2c00f7810, /dns/validator.df59cdc3-477e-4652-b5f3-f364fa423833.aptos.bison.run/tcp/6180/noise-ik/0x58bd29a36a105bcd5cc022c4fe4d9c164e4283254f2735412f74f1c44e868955/handshake/0, outbound), (8d7766a61a01a33199db8412362805e9550d7dcdce6ab74fecb2ddf9899f74d0, /ip4/65.109.38.111/tcp/36512/noise-ik/0x26100844272e2371b6beefa5e6001e9175656e378c7cfd2546aea8a64768a421/handshake/0, inbound), (234c957655cc9f4a5f8f0d1ca8f99baf8af3ea57740cd71fa2ab3d14fa3a4ede, /ip4/88.99.216.174/tcp/45776/noise-ik/0x984d5c5cfeab5919f5d58e304d3528b609f828ea51f5609ef5e4bcdb759d573f/handshake/0, inbound), (77700fe6d6dd6dca2c8ca582f05ab1e3d803034d8ab1563757275a1720823cae, /ip4/34.100.202.152/tcp/59284/noise-ik/0xec30e24b1847479ac7d43646e8c43360cb82f5def59c1843b25aaa4446c43613/handshake/0, inbound), (d947abce2dab0abd4e0b705db96db38e0444c7347232ffed1b4d29cca74f86ef, /ip4/13.234.80.181/tcp/60864/noise-ik/0xa25316649c31e35ff7373f8e4361ce3bf1a0940034db6b7143583f58d0be756c/handshake/0, inbound), (b404626788b5ed7e6940200acfe6884f91cf636c62751bd12508d51bdb80e13a, /ip4/92.255.176.91/tcp/50927/noise-ik/0xabac71d7faba0e2d26daffb8365b12706f6a09d24805b98259e197b1b30c9429/handshake/0, inbound), (a9e136449be1ace0702284b34da3ce1742231cb844c1848c2ea95e2fc9b18ecd, /ip4/178.170.41.40/tcp/52068/noise-ik/0x4da04f388b8428746842a969cc8a1515eb3eea08705bee0da41c60370709f72e/handshake/0, inbound), (ef5bbb8237eeac0d4d1764f1802e123ae3d3deacc42d39a2e0884a0245263c32, /ip4/34.176.106.60/tcp/57404/noise-ik/0xb8369d79e36003f8a57bda1bb4dca9509eeeae1e4464753f0be94745e90e5c66/handshake/0, inbound), (62562701d5f396a2f8e7bc44e8700214d0cbc38c46d78dbf9dcab949f1dec6c5, /ip4/138.201.52.102/tcp/36338/noise-ik/0xb47c4d62a02edf58248ef79b90e441f8d1a691e0f0a86a62015c332f21884b59/handshake/0, inbound), (5e7ed6a3d40cb5c8831edbfefb4bbfd4874e5566556eea08dde72c14514af7b5, /ip4/131.153.1.130/tcp/45148/noise-ik/0x6494a2c60b596dde68a0a37191c5f88e901646fe0225c3008d174a919f6ee472/handshake/0, inbound), (71eb495f8fb7d8f9912dc7ba1406559ad6a7fa54d8abb2bdf5eac78184719798, /ip4/88.20.151.167/tcp/33022/noise-ik/0x446627da30aa8ca8a3da22d0211f5a56699fecbbc39ad46acc2a3fb4b315b125/handshake/0, inbound), (a89b6feeccb1e459737a3e0bd69628b15aa795f68deb2a113cd9177c86984c39, /ip4/116.203.125.179/tcp/34578/noise-ik/0x068416ae83ca031ae8e15c8584286a8af7d4e690dd73cd280e091d58be73b026/handshake/0, inbound), (b77026ce272a63b7261d20e5d0d9ca8cddd81b42b3432891668c43c03dbd1b73, /ip4/3.133.88.245/tcp/51162/noise-ik/0x3829ed5aabd623fdb814ff22a62293630f1736dfb7a5d5fddcdf8fe4c59e892c/handshake/0, inbound), (62c89435fbbe74628844c05c3429e4116a3c4fc0a276a25901f7c40432f31925, /ip4/65.109.17.85/tcp/58426/noise-ik/0x693670aa97d37817bd7059305e0ff517988d5f2eb6fa6938a959d7e5e0b43d73/handshake/0, inbound), (94d9705a8a3333eb52c96889c50e6e02294d7c55ac24b3d327dd2bb79aff1974, /ip4/34.125.177.4/tcp/58990/noise-ik/0x457a5aea815638b2a0e1d214269b76b2dbe92f1e200bb8d44dcbcaf8cf1f7074/handshake/0, inbound), (bb9451dce59fc99144c23c37ce0adc50fc93c2b50ab316fb4aeaa83647244bc9, /ip4/95.214.53.37/tcp/44712/noise-ik/0xaaf7d34462c2728ea4f1c6d1b4e112f3fee28df76ae2be6340e30ea7a14c0629/handshake/0, inbound), (d78ecbd74f9e224a80374ca807e480c4d24f88d0599a263e7949cd7129dff377, /ip4/158.247.232.85/tcp/55514/noise-ik/0xe0b694dc10f726fc863614aa0e37dbafb57f5b10f145ce1ced1b4e841ebec32b/handshake/0, inbound), (e8278487a145eadfa3b07d7a711efe1606a7184045337b27259d6ab73e89aa5a, /ip4/65.108.214.159/tcp/48864/noise-ik/0x4f445a49835f6e39a0a11ad1ffe5d11cd4155e03a0147a18d73caffa02b3f32c/handshake/0, inbound), (d6e173c7029f5af061f458062cfdb5a57f78b528675c988df9fa11b7c72130e2, /ip4/80.190.129.50/tcp/7135/noise-ik/0xa832001ea77812189b50c9fc0efb567e50e6b34e47623cdab0e04d714bf2597f/handshake/0, inbound), (394f58ff583315bf52b055d34c8635329f4426626b57da5c709a81aa6a68d83a, /ip4/65.108.67.152/tcp/34188/noise-ik/0x837b9db86b66e0a2bbaa6d6122cc376fa7ed2657b44d63cb37d546d435ae3458/handshake/0, inbound), (4247098897780ab7b0c02864dd8ebbeb4dbc0df85c03f82edbb0c00a78fce685, /ip4/65.109.38.180/tcp/37530/noise-ik/0xca05e1e1c7ba782d0cfaade4a66e9fb0e524dae7cd8ffddbe053c0c0cf7e5a78/handshake/0, inbound), (06b15cee0fddb86956e883d0156eb7b5da441e7139ea00ef39bcf0cbf9dc52b5, /ip4/213.133.98.80/tcp/49116/noise-ik/0x6446772efcc33eb8cc2fa338616a16ad506f4b0cac731c9f2bb36e24b8ba8c2f/handshake/0, inbound), (5eea1639b99765ff96502180949fac3662ffd89d358e4adff5dcd31b60ed72c0, /ip4/34.215.68.7/tcp/39462/noise-ik/0x899a351fe3c93e6155143236649f13d3b0f140e48d61ab03b091b06e340ecf6b/handshake/0, inbound), (ae91b5f2c5a13f5394c46bcfc7e67e4db1b3659dc6c6faa08e4b9a76a8dde53d, /ip4/185.92.222.137/tcp/43186/noise-ik/0x77e7ba11b534329d600cb3b20223491a27a1a585a860b319c8f38f01985a5b7c/handshake/0, inbound), (e38cec9883de5eaa58452d4860e13efabeb986876a7e51cc0e055f540f1a9bf4, /ip4/47.251.0.151/tcp/52304/noise-ik/0x9874eda5137ce8d63df3f0644fec1ae9bcb2a542f76b77575d5436d483cdbd15/handshake/0, inbound), (e0c9b84c58ba4234aeebbb7203746c6f275025ceee94bb86d4573289565bbdde, /ip4/142.165.207.19/tcp/33900/noise-ik/0x4f0ba2280e4c7b1911518db0e666b9da4f829a9b9201e8abfb6e48502c0cbb13/handshake/0, inbound), (87a572ffb4a0c22b32d41b91d1df342da0d4477b234faa394f0328bed498a1be, /ip4/13.215.49.17/tcp/42738/noise-ik/0x7fda3b73314250e18c307b9e5703db316f4658309bdf4478cc7b1e4becf8d077/handshake/0, inbound), (44c6dc9efba8be28f10f8c44cd318c5656721c1d88ed0e8e5a3609ac8ed4d63f, /ip4/206.189.32.169/tcp/58510/noise-ik/0x6dd9076dbb823406bb448b40b1a50873819941467ea66502e952794bf0bf2b56/handshake/0, inbound), (1614c0510f246f089e63cb0dbf1c30529d6389d4b6591d209afd821065bf3541, /ip4/89.149.30.220/tcp/56024/noise-ik/0x55c78feb9ab676cb5049b00630a92c161e93d1f68f82d62bd8b405ac6ea5fb54/handshake/0, inbound), (4428fefbc147d0170f9bbdfd7310adb642f9a50fcf7df338e4a95f2ea2033ed2, /ip4/5.161.157.104/tcp/47156/noise-ik/0xa6a942b25ccfb3684cd21923c8b3716489f3a4626153627add4dd6860b8d9e34/handshake/0, inbound), (45cf704e3552993db28b0235fec04bc279b659f3eb64269f93f2f7a6c9bd8ce0, /ip4/185.32.190.102/tcp/51708/noise-ik/0xc06f9d8d2ea6759a18ae2828817114db3df4f554f761f3b9cdd2d4ae89ee654a/handshake/0, inbound), (8d1f8a22eb447833aa978f611c13d0fa327bdda2bed8ca169b4417edbbdf4aaf, /ip4/18.136.12.20/tcp/49064/noise-ik/0x5efeff93d9ba09505c26d0bbde3cfa6362147171f6525151569ae1ade964a849/handshake/0, inbound), (943bdde7612aecb92e745facfdf732b2392b1d27f9fd852d83d0f8619fd49382, /ip4/65.109.38.49/tcp/58266/noise-ik/0x0d14f67b1647f658bff096f1db8b8c999b2119f83f721f072e692772efba4a6b/handshake/0, inbound), (dcef38c93319415362a06c921481e70887d87d0650bf31ad40fc858d72d80cb5, /ip4/206.246.84.36/tcp/46016/noise-ik/0x04ee34dc24955ccf95a859f30da201c29de57e7fe77fe4ea253aec904d62235d/handshake/0, inbound), (afea364425825255938bbfdb1704f06ec56d5c914833208b96fc47b94aff1671, /ip4/5.188.4.254/tcp/35646/noise-ik/0x2948ff5016e8abc49d453f94fb12adf2c77671e85c68665d56a2b75a54bc2e09/handshake/0, inbound), (b41568279f81fccc440bdbb807f68eb0cbdd494c74de5cccf545b0a47d890e93, /ip4/34.95.49.24/tcp/34440/noise-ik/0xbfeed27adcb200fa2cca268caacd88a39ff0c3cc386f6ec8d23ff7a30257ba3c/handshake/0, inbound), (08c3f260d82e0aa572b0f58eb85a6edeb9ca5022fab829a5fb3a21952710f63f, /ip4/195.54.41.148/tcp/51890/noise-ik/0x204360f8880bf7faf61ee57ebc910f8adfe8326f6a78ca4217ec4f4e6a8c9a05/handshake/0, inbound), (b7aa84d8be549026574559735403a79b001ca2c3be54d405b8ec7c4df79c1b15, /ip4/34.123.186.220/tcp/1095/noise-ik/0xfb37ed6b0ee35d4745c306260bb93dbcbb634d13c9de2c2dad671ce50258fa1c/handshake/0, inbound), (8c91f8c74f48784f9a5c873d372e52bfbce6fee6ae420e04ef346e9bffae9d63, /ip4/44.201.171.170/tcp/39086/noise-ik/0x0e8ffc16f7396cb8ae7ca3b6a87aa614f13b6fe767613bcf2ac2cb41ce06af0d/handshake/0, inbound), (202c77e0e8b6f7c17428c989d3138a75c989e7fa61c1bb3d38fdbbef0b5bca79, /ip4/38.242.139.6/tcp/33416/noise-ik/0xf9bf277359939f74f30d2e632f25f18e0ce3a1f5997fb65f11362f300177594f/handshake/0, inbound), (0e41c1afa2edc48280f24f0b456311086fa3f777f25b2db7d9d12a91526214a2, /ip4/20.168.103.85/tcp/40042/noise-ik/0x0260f224f2f8025757190be9f4be422a38172e7f5356bfa5482758617c208565/handshake/0, inbound), (e3370ffd180416064f3c8e529da8cbe14e2567506834d8d90be3ab7fbb50ef6d, /ip4/178.72.89.150/tcp/50786/noise-ik/0x95c784883909c3e315c7d2bb3626fa2b375d10bd26905645fdadc7ff0973693f/handshake/0, inbound), (86b5e89746c45648735ef61c722debec0c35e051db941c783a82756d77a54f8c, /ip4/147.182.152.64/tcp/47450/noise-ik/0x4a070f19419fd9db4d8ed2c1232f21ac356fccd4c5fdcfb232a82f98e6bb3c0c/handshake/0, inbound), (d52684b0f57370239beeac25d0c9ad1db6e224a96c725eb5ffbc070231f31cc8, /ip4/5.22.216.173/tcp/60120/noise-ik/0x0233a759e49ce4e0f0b6f37ba709147bd99e2329599666d33b81b1ec51b6fc66/handshake/0, inbound), (7ed9a1107545401206e1ae1758fa2ad86e2772c49a0e0ca7383286e3e81a8e34, /ip4/149.248.15.167/tcp/43452/noise-ik/0x08d20490635a3879126fbc4063938876d3d36874d2bb22dbe62ead97607e5c01/handshake/0, inbound), (5be2700c076c27678184fad6af04a0541fd0f2f1ed66c49cca5fe16a466807f9, /ip4/35.198.29.93/tcp/1125/noise-ik/0x6e6f81630802697679e84829fcb0a29981a7eb501c16ce396b234ee7929cd926/handshake/0, inbound), (c310249c3f3cc990e95b6dfc4da612373e66fd300d707390b4719665a1e7e52c, /ip4/161.35.209.175/tcp/44818/noise-ik/0x409bee10bfe882f66da553f27fbd1c81b0121ba3486337ce81e0faa220a43466/handshake/0, inbound), (631bc1f429ca72def1ebb108c442211b6a2e9337188342ef7b1eb132aa91efc4, /ip4/65.108.76.44/tcp/35362/noise-ik/0xb5d0e7c8028cb94d6bc1ffbf89ed133b3993b9db94f7b4c69b307c6ecef10a10/handshake/0, inbound), (07d7df40b1b25dcb7e8fb6caa76ee2272f081a8992629e145aead916b8713f3d, /ip4/65.109.38.34/tcp/40152/noise-ik/0x327a382397ddd711345a5655cb42e4ea1ad1e3fa9d0d2ef03eb3db21901ea30a/handshake/0, inbound), (b9eab269389f080a221e1811493284ff7141b542dd2bdcaf4fe3e4f4518e8f43, /ip4/204.186.74.42/tcp/40656/noise-ik/0x90b85e9b45915c9160569b53abff0756637873fcfc72c376d2fd4a9db9a6de43/handshake/0, inbound), (27cf77d1b6516f9616f8cea50d1667dc7a234724983a8f06d179260cea3fb165, /ip4/209.145.62.91/tcp/41954/noise-ik/0x01ea5dd19128622708b819b3b380fc764f5aa2afd7624cf1edbd8419144ac07e/handshake/0, inbound), (87a92d3c58a7b644f0efc520f210f590573d10fad04ba9aa0ccb343dfdc36439, /ip4/35.203.107.196/tcp/43112/noise-ik/0x3c1cd1f7be2d6575700a78f685b3dfec2ce04fb59c710174f070045bf1727d20/handshake/0, inbound), (4b904c781e92c118f45b3bd4897fdeb769cd1b638c81841c8bcdba98985698ee, /ip4/52.55.23.138/tcp/43128/noise-ik/0xde91eabc9016a46cc9e97ad0640cce1e12ed59ca4e06c3e3bcb42cd74d570b7f/handshake/0, inbound), (f4cb45b3d657a4957d723cfaaeca5803c91ea38981bd306abf7a8c44790c894f, /ip4/65.109.38.218/tcp/45410/noise-ik/0x6cc5aaa3ba335a87716a4415cc887160376cda5b784c39b2bbceb5891ae41915/handshake/0, inbound), (9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1, /ip4/167.235.56.58/tcp/45468/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0, inbound), (da26d0b97726a315035486da1562a1e7d42c1f8bd50fee123402a1e5d4e3af92, /ip4/67.219.102.185/tcp/58976/noise-ik/0xe600704be1c9d36e33e52622298bc235b40a24629dd080c38a4b257af9479678/handshake/0, inbound), (e5207ae13fce7f9cfdf3e388e415d97cbf55de727d0e7dcb97e7ca2f209fff76, /ip4/188.34.194.94/tcp/39952/noise-ik/0x17d2780a50ed9c68d1caa037438c22d2c3afdfe7910f86f272a0f8e0e3724637/handshake/0, inbound), (26a5f68e39a2543aef7ceaec7dcd047510d856b495f95979879bd278e4725e07, /ip4/20.216.33.211/tcp/50230/noise-ik/0x82adaf4d3d0989bf6bbd89ad65ebf377639212e8ef9625dd8440e61328bfb12c/handshake/0, inbound), (5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094, /ip4/211.91.254.188/tcp/55164/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0, inbound), (337925eaba30d718f87ddc251c8c3314322f8e0bb87cf2e6c77cc4211b97e493, /ip4/45.142.215.220/tcp/60792/noise-ik/0xd87250f1178ab554468d6ce84c92e5fd2584863466c79c5b6983f9e54822ea11/handshake/0, inbound), (9eaa6c899597ca4d691cf453a1b0b17b534ed8fa91eac78a9e9bc0f457ee8cba, /ip4/154.12.228.182/tcp/56448/noise-ik/0x87f0aa2ffc47815660d1319275e34fcb90ed7e71e4de91430bf4facd809b8b4f/handshake/0, inbound), (87b30b548452c5a3200a1aec4251a1a01b05b15cb74d41e49701665ef4d36804, /ip4/65.108.157.108/tcp/39460/noise-ik/0x94b911924410fcfca4e9727a575fc240c52c890cca096424e045b70215f8f440/handshake/0, inbound), (b178f79f9bb08355fad69e6ddd7cd5daff672e89c6f8f1921fa03d53adc52e46, /ip4/154.53.45.247/tcp/59684/noise-ik/0xc7ca90ca3bc293f61047ffe4b630bafdf4a457970cd861fe773bcaa43c59d87f/handshake/0, inbound), (85c351605b52d46c016e04876239647644bf1a83619f8ff6c0b36f3f344d79ff, /ip4/65.109.39.167/tcp/50656/noise-ik/0x7cdf178d1a4e58f3d4139ab0a7089e15c8c8d7bad757e458dc1dc803e4d4500b/handshake/0, inbound), (fecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6, /ip4/80.211.33.21/tcp/50110/noise-ik/0x25b580f3600c04c475394b43ff7a333fc88aba5a9b75448f612b7b67bac97475/handshake/0, inbound), (a4079618c9f4cbbf11d69d7a12b7f5a6deadef4aed991c00e4fd412e2765b713, /ip4/202.61.225.117/tcp/47750/noise-ik/0xfdeb561fc0601031a6b1faa1e1c2053e24aaa6f9c44e07d926dadd2100602705/handshake/0, inbound), (0e3557dd416faf22f7efa010cbaf9f47d40cdfdf98d2fe26145ec01b3d45c2c0, /ip4/94.26.228.247/tcp/57012/noise-ik/0x5c0cca62689aa412b09080a18c74d9f63f813c80acb6375e54279215e925f656/handshake/0, inbound), (b2243c6234285ca4ddba4927603750a0b2571d9453b1021c4cd4a0c34e045ecd, /ip4/88.218.191.79/tcp/55276/noise-ik/0xd87ad6964b3881a5846e134e6506db87b24147c1c3c776991fe6af7b84838f49/handshake/0, inbound), (c9b19c9b6afb63f3e9fd8c0491fa6ca9fa6a008cd5f2b2dc3b5d7a984b9c5a7f, /ip4/217.13.223.167/tcp/39936/noise-ik/0x4e8c921ae3f525047ed362170a6e90e1656a4f0734e04d4d4a4f3c32013ffc4b/handshake/0, inbound), (568e99f8422a6a49d7e648db5e58747103e8495989e2a4680faa5cbeb296a4ca, /ip4/115.188.162.86/tcp/47716/noise-ik/0x20553805dd22871cf94bcee8ec4539345a83e1c4a1e129da9644e7940cfc5a69/handshake/0, inbound), (6f929de7b48e002849eaa3aff478dc8ac427283dc185891c457180b7395c5f16, /ip4/5.161.148.93/tcp/35718/noise-ik/0xa4521f479dbdee80bef59eb3fec165e55e0fc32db51bc10ba6c86a30c50ab548/handshake/0, inbound), (5bd7c59fc0f31a87ab9633a228fd7a9d6b8081a792abb688047621cdce99b012, /ip4/202.182.103.182/tcp/41842/noise-ik/0x02b16f3837014afb50805e96285d5b3685abbaf10e0e1b753da19b62527c4413/handshake/0, inbound), (e342f41235dcfed6016769285e5f918530e9dbe6832e553df653e050d7347168, /ip4/211.42.152.211/tcp/54516/noise-ik/0x0018f478677a05782247b23cd12aa14c6ab5e25ae78e296056c32d585e0d1b27/handshake/0, inbound), (1dd17527c01616b252c9801366c158fbc0ff67b5ee76436e3cec04399d8de5d0, /ip4/167.235.26.108/tcp/49772/noise-ik/0xa6bf1ee023a3a51c554752e9db07d554a69c36e7938c5404d4e33917f9004a5c/handshake/0, inbound), (5a933bb07b7fae5cc0f5609bc9f854f1bb02c6a15319c1fd67f988d5d07e3ec2, /ip4/44.225.105.186/tcp/10368/noise-ik/0x54350f682ba782768b6fb3708f9354575a4b2e9c2f15ce5f202c2659905dd371/handshake/0, inbound), (21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42, /ip4/95.216.35.43/tcp/55730/noise-ik/0xab428f209b72d38b192c2c0fa2b3df1219fa5b509741e2c25529e261e9d6cf26/handshake/0, inbound), (8089a7bd4bbdcd6f0ba7812313b49e99f569880644e8f4ad9fd3ce584e0c22c4, /ip4/35.247.231.123/tcp/48608/noise-ik/0xef29166e7328e0b8a9db5ed1f34afef510cd39f79c53cd2dc24b2380231e3d36/handshake/0, inbound), (57be1bdaf544178f3e754dd23f84c74a95785c694066f6c1d48762b8c888cbee, /ip4/159.69.84.105/tcp/39996/noise-ik/0x321541d5856085fc6a15093c2eaef043a96bbe0e5b0a403c2d4b7db37c615306/handshake/0, inbound), (69268e38beda87f1ff5bf22881f22b9deb564c89028e7af89ee8b911fc616f95, /ip4/34.142.153.143/tcp/1065/noise-ik/0x15af392cd632f94d117cc8d16c5864dd536c095683fae699b8080b23cb1f9451/handshake/0, inbound), (840880fc27f01706d134397dcbeb14cd54899fc8dc93f3f9f1b1a454840829fe, /ip4/54.183.25.175/tcp/25868/noise-ik/0xd1c824985d4a2bd12419ea84973852829af6043bc9912d3275af48d2a2bcf323/handshake/0, inbound), (ae050e3c60e8f4e7817e0aaed4007e64a3d10b713f85f7ff28a7a7fc0d3f80e8, /ip4/158.247.223.170/tcp/59700/noise-ik/0xab8aa93b079317a9309e806d009ce878c745820f0fa0c9738786ab4f050b520f/handshake/0, inbound), (ae289fd9643d98f1c486bbf983a0400043206c9e84c920957487910988817a78, /ip4/65.109.48.174/tcp/38606/noise-ik/0xa76ec55b1997bbd65e8a9968dd32f541073081666386667e6108716d0d0d0d78/handshake/0, inbound), (f2e802753bbaa2b3e0861180dfd2dd3fe8dc9bb0d49e0094376f6d565bab9884, /ip4/89.208.142.102/tcp/49200/noise-ik/0x2423b0e44a819fbbf345e147c79d9202151855031e3cc4eddd5de2e304286605/handshake/0, inbound), (33e6210e459009bd91061a50355ad1837c2467640a5c50eec23fbc223d9ae788, /ip4/187.150.154.239/tcp/43426/noise-ik/0xaef2f5cde7ac7d8ddf44b9d6a9ffe69c436d18ea3ed1caed2db3930d1ad4154b/handshake/0, inbound), (f758b0bf7a9bfb700b84b83779bb92ef8d7a4c22587bf7a05481abd77713df65, /ip4/173.249.59.200/tcp/42272/noise-ik/0x9b5d0db8bd43ea7df624a98227785cd11aa9acc0b15702789cb248b818160151/handshake/0, inbound), (89d079d0935ed64d8a755bfe97938150bd56131336d656b31db0eacdbf03b545, /ip4/34.176.59.35/tcp/1137/noise-ik/0x67f380e6f00c16637c4f284b82182d92518069cfbd725116d187ea534ec0aa2d/handshake/0, inbound), (b92890f0c60eaa2b79814615402ebbaf6371052e296049f1788870a9887a2dc7, /ip4/104.238.221.174/tcp/59416/noise-ik/0x92aedafd2e0a233079fba8925f62fb61f8bf1a03c53e33c70b99ce2a2cb16208/handshake/0, inbound), (5badd632abd92d673892e42578151881d644472326b6c021603f1110f10f74b1, /ip4/65.109.39.154/tcp/54308/noise-ik/0x92ba0616498c5050a783d65f7c8565ffc339e54f3dfca54d460f3069fdf6c165/handshake/0, inbound), (d2f842ded238f6ba4629650863dc3ebbd506f857c6e1050a4ac923fc60e2e4a1, /ip4/216.238.98.187/tcp/41586/noise-ik/0x7c703573f3baee0acff1907cdf0c47abf96fa9bd84e1b6ef7a856f94d2202905/handshake/0, inbound), (975757242ebe7b3b679b25795fa93b1ed07521003c007faf0e53708a4a0a2bbb, /ip4/134.19.231.7/tcp/44302/noise-ik/0xe5556e5c0b894bf121c63e1b404e80db325baf704680ac971597c0b7bfd57011/handshake/0, inbound)]"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:626"},"thread_name":"network-vfn","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.598700Z","message":"Current eligible peers","data":{"discovered_peers":"DiscoveredPeerSet({})","network_context":{"role":"validator","network_id":"vfn","peer_id":"00000000000000000000000000000000d58bc7bb154b38039bc9096ce04e1237"}}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:582"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.613617Z","message":"[validator,Validator,03c02943] Dialing peer c3f2c9dc at /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","data":{"network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
validator_1 | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.668795Z","message":"[validator,Validator,03c02943] Outbound connection failed for peer c3f2c9dc at /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
validator_1 | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.668931Z","message":"[validator,Validator,03c02943] Failed to connect to peer: c3f2c9dc at address: /ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0; error: Transport error: Connection refused (os error 111)","data":{"error":"Transport error: Connection refused (os error 111)","network_address":"/ip4/179.48.251.14/tcp/6180/noise-ik/0x33d0ec1fdd859302ef108c7521f77d028d188432555cb66b8e78354e92921352/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"03c02943185bd29f525828fc418a80cf2efb1da7a8e44d7d06e045422176b824"},"remote_peer":"c3f2c9dc502dbe9de4d3d3ee472fa679f578d978e53c1242a6906f894d435bd1"}}
validator_1 | {"level":"INFO","source":{"package":"aptos_mempool","file":"mempool/src/shared_mempool/coordinator.rs:342"},"thread_name":"shared-mem","hostname":"737654cafb21","timestamp":"2022-09-01T00:46:10.702376Z","data":{"event":"live","name":"g_c_runtime"}}

I got some this message warning about tokio libary, why my node get many tokio?

my node status :validator
I never upgrade to full update

any idea?

{"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"d50b969f5dd2","timestamp":"2022-09-01T14:50:57.393034Z","message":"[validator,Validator,87b30b54] Error making outbound rpc request with request_id 4 to 5bd7c59f: Error: Rpc timed out\n\nStack backtrace:\n   0: network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:477:57\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:490:37\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n   1: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17\n      futures_util::stream::stream::StreamExt::poll_next_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9\n   2: <futures_util::stream::stream::select_next_some::SelectNextSome<St> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/select_next_some.rs:34:36\n      <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      futures_util::future::future::FutureExt::poll_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:562:9\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n   3: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:290:13\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n      <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/poll_fn.rs:56:9\n   4: network::peer::Peer<TSocket>::start::{{closure}}\n             at ./aptos/network/src/peer/mod.rs:254:13\n   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n   6: tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n   7: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::LocalNotified<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:376:9\n      tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:435:13\n      tokio::coop::with_budget::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:102:9\n      std::thread::local::LocalKey<T>::try_with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:445:16\n      std::thread::local::LocalKey<T>::with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:421:9\n   8: tokio::coop::with_budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:95:5\n      tokio::coop::budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:72:5\n      tokio::runtime::thread_pool::worker::Context::run_task\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:434:9\n   9: tokio::runtime::thread_pool::worker::Context::run\n  10: tokio::runtime::thread_pool::worker::run::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:386:17\n      tokio::macros::scoped_tls::ScopedKey<T>::set\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/macros/scoped_tls.rs:61:9\n  11: tokio::runtime::thread_pool::worker::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:383:5\n  12: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:362:45\n      <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/task.rs:42:21\n  13: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n      tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n  14: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::UnownedTask<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:413:9\n      tokio::runtime::blocking::pool::Task::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:91:9\n      tokio::runtime::blocking::pool::Inner::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:325:17\n  15: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:300:13\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18\n  16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:505:17\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      std::thread::Builder::spawn_unchecked_::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:504:30\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5\n  17: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      std::sys::unix::thread::Thread::new::thread_start\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys/unix/thread.rs:108:17\n  18: start_thread\n             at ./build/glibc-fWwxX8/glibc-2.28/nptl/pthread_create.c:486:8\n  19: clone\n             at ./build/glibc-fWwxX8/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"87b30b548452c5a3200a1aec4251a1a01b05b15cb74d41e49701665ef4d36804"},"remote_peer":"5bd7c59fc0f31a87ab9633a228fd7a9d6b8081a792abb688047621cdce99b012"}}
{"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"d50b969f5dd2","timestamp":"2022-09-01T14:50:57.393077Z","message":"[validator,Validator,87b30b54] Ping failed for peer: 5bd7c59f round: 1547 with error: TimedOut","data":{"error":"TimedOut","network_context":{"role":"validator","network_id":"Validator","peer_id":"87b30b548452c5a3200a1aec4251a1a01b05b15cb74d41e49701665ef4d36804"},"remote_peer":"5bd7c59fc0f31a87ab9633a228fd7a9d6b8081a792abb688047621cdce99b012","round":1547}}

As the title, After updated form testnet_ 8141a975f9e7d831e9699c58a83724ac9635d991 to testnet_ 26ed95ce4c8798743e7900bba566d29079af0a70, the node failed to start
Tips: the node has been normal until now

Validator_testnet_26ed95ce4c8798743e7900bba566d29079af0a70.log

My owner wallet 0x27cf77d1b6516f9616f8cea50d1667dc7a234724983a8f06d179260cea3fb165

Node can't sync for 2 days and stops periodically. I have tried many options, but no solutions help, except rebooting the node, but this solution is only temporary. Naturally consensus_proposals_count 0 and respectively
REWARDS PERFORMANCE is 0, but Liveness is 100%. What else can be done about it? It seems I've tried all the options in these 2 days.

My server meets the stated requirements, but I wonder if I can run another server with the same keys, in approximately the same location, or would it be against the rules?

Now I think there may probably be a problem with the disk, although my provider claims that I have the latest SSD. I'm trying to find an opportunity to replace with NVMe.

Hi team,
@sherry-x
My validator node is syncing to about 98% and stopping.
I tried to restart it 2 times and it crushed.
Internet bandwidth is 100/100.
Average usage was 8-10Mbps and it dropped when reached 98-99% of sync
Log is attached.
Thanks
log1.txt

I have 16 CPU; 62 GB RAM DVS on Hetzner AX51.
Set up up with docker (offiicial guide)
OWNER ADDRESS: 0x21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42

consensus_proposals_count increasing very slovly and no rewards for last 3 EPOCH and reward performance it still 49%
LOGS

testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:384"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.415732Z","message":"[validator,Validator,21b4ef9b] Closed connection to peer: 28dde00b","data":{"connection_id":294,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.616514Z","message":"[validator,Validator,21b4ef9b] Error making outbound rpc request with request_id 6 to 9f9ae72a: Error: Rpc timed out\n\nStack backtrace:\n   0: network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:477:57\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:490:37\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n   1: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17\n      futures_util::stream::stream::StreamExt::poll_next_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9\n   2: <futures_util::stream::stream::select_next_some::SelectNextSome<St> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/select_next_some.rs:34:36\n      <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      futures_util::future::future::FutureExt::poll_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:562:9\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n   3: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:290:13\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n      <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/poll_fn.rs:56:9\n   4: network::peer::Peer<TSocket>::start::{{closure}}\n             at ./aptos/network/src/peer/mod.rs:254:13\n   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n   6: tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n   7: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::LocalNotified<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:376:9\n      tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:435:13\n      tokio::coop::with_budget::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:102:9\n      std::thread::local::LocalKey<T>::try_with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:445:16\n      std::thread::local::LocalKey<T>::with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:421:9\n   8: tokio::coop::with_budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:95:5\n      tokio::coop::budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:72:5\n      tokio::runtime::thread_pool::worker::Context::run_task\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:434:9\n   9: tokio::runtime::thread_pool::worker::Context::run\n  10: tokio::runtime::thread_pool::worker::run::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:386:17\n      tokio::macros::scoped_tls::ScopedKey<T>::set\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/macros/scoped_tls.rs:61:9\n  11: tokio::runtime::thread_pool::worker::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:383:5\n  12: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:362:45\n      <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/task.rs:42:21\n  13: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n      tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n  14: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::UnownedTask<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:413:9\n      tokio::runtime::blocking::pool::Task::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:91:9\n      tokio::runtime::blocking::pool::Inner::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:325:17\n  15: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:300:13\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18\n  16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:505:17\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      std::thread::Builder::spawn_unchecked_::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:504:30\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5\n  17: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      std::sys::unix::thread::Thread::new::thread_start\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys/unix/thread.rs:108:17\n  18: start_thread\n             at ./build/glibc-fWwxX8/glibc-2.28/nptl/pthread_create.c:486:8\n  19: clone\n             at ./build/glibc-fWwxX8/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.619311Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 9f9ae72a round: 31 with error: TimedOut","data":{"error":"TimedOut","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1","round":31}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:421"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.619563Z","message":"[validator,Validator,21b4ef9b] Disconnecting from peer: 9f9ae72a","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:350"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.619830Z","message":"[validator,Validator,21b4ef9b] Closing connection to peer: 9f9ae72a","data":{"connection_id":279,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:370"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.619842Z","message":"[validator,Validator,21b4ef9b] Connection [9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1,/ip4/167.235.56.58/tcp/39520/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator] closed due to Requested","data":{"connection_id":279,"connection_origin":"inbound","disconnection_reason":"Requested","network_address":"/ip4/167.235.56.58/tcp/39520/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.619994Z","message":"[validator,Validator,21b4ef9b] Error making outbound rpc request with request_id 5 to 5e841137: Error: Rpc timed out\n\nStack backtrace:\n   0: network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:477:57\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:490:37\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n   1: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17\n      futures_util::stream::stream::StreamExt::poll_next_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9\n   2: <futures_util::stream::stream::select_next_some::SelectNextSome<St> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/select_next_some.rs:34:36\n      <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      futures_util::future::future::FutureExt::poll_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:562:9\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n   3: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:290:13\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n      <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/poll_fn.rs:56:9\n   4: network::peer::Peer<TSocket>::start::{{closure}}\n             at ./aptos/network/src/peer/mod.rs:254:13\n   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n   6: tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n   7: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::LocalNotified<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:376:9\n      tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:435:13\n      tokio::coop::with_budget::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:102:9\n      std::thread::local::LocalKey<T>::try_with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:445:16\n      std::thread::local::LocalKey<T>::with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:421:9\n   8: tokio::coop::with_budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:95:5\n      tokio::coop::budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:72:5\n      tokio::runtime::thread_pool::worker::Context::run_task\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:434:9\n   9: tokio::runtime::thread_pool::worker::Context::run\n  10: tokio::runtime::thread_pool::worker::run::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:386:17\n      tokio::macros::scoped_tls::ScopedKey<T>::set\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/macros/scoped_tls.rs:61:9\n  11: tokio::runtime::thread_pool::worker::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:383:5\n  12: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:362:45\n      <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/task.rs:42:21\n  13: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n      tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n  14: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::UnownedTask<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:413:9\n      tokio::runtime::blocking::pool::Task::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:91:9\n      tokio::runtime::blocking::pool::Inner::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:325:17\n  15: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:300:13\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18\n  16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:505:17\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      std::thread::Builder::spawn_unchecked_::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:504:30\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5\n  17: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      std::sys::unix::thread::Thread::new::thread_start\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys/unix/thread.rs:108:17\n  18: start_thread\n             at ./build/glibc-fWwxX8/glibc-2.28/nptl/pthread_create.c:486:8\n  19: clone\n             at ./build/glibc-fWwxX8/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620073Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 5e841137 round: 31 with error: TimedOut","data":{"error":"TimedOut","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094","round":31}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:421"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620111Z","message":"[validator,Validator,21b4ef9b] Disconnecting from peer: 5e841137","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:802"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620125Z","message":"[validator,Validator,21b4ef9b] Removing peer '9f9ae72a' metadata: [9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1,/ip4/167.235.56.58/tcp/39520/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator], vs event metadata: [9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1,/ip4/167.235.56.58/tcp/39520/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":279,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1","stored_metadata":{"remote_peer_id":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1","connection_id":279,"addr":"/ip4/167.235.56.58/tcp/39520/noise-ik/0x4df1d67ecd21fff7647ca8abf9ccb3264b068e111fe03bc7def455a882dda30a/handshake/0","origin":"Inbound","messaging_protocol":"V1","application_protocols":{"inner":[231,152]},"role":"Validator"}}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:350"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620274Z","message":"[validator,Validator,21b4ef9b] Closing connection to peer: 5e841137","data":{"connection_id":280,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:370"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620285Z","message":"[validator,Validator,21b4ef9b] Connection [5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094,/ip4/211.91.254.188/tcp/35442/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator] closed due to Requested","data":{"connection_id":280,"connection_origin":"inbound","disconnection_reason":"Requested","network_address":"/ip4/211.91.254.188/tcp/35442/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1  | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620308Z","message":"[validator,Validator,21b4ef9b] Outbound connection failed for peer d801e1b2 at /ip4/95.188.77.87/tcp/6181/noise-ik/0xcc98cf612f641136ff744bcfd0a3228219fa815e9269556b6d992c23236b462f/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/95.188.77.87/tcp/6181/noise-ik/0xcc98cf612f641136ff744bcfd0a3228219fa815e9269556b6d992c23236b462f/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"d801e1b2c79395df00ff26f63f60e2f11d541d18d6755831ec1a6537fd3ad6d3"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:384"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620389Z","message":"[validator,Validator,21b4ef9b] Closed connection to peer: 5e841137","data":{"connection_id":280,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:802"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620456Z","message":"[validator,Validator,21b4ef9b] Removing peer '5e841137' metadata: [5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094,/ip4/211.91.254.188/tcp/35442/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator], vs event metadata: [5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094,/ip4/211.91.254.188/tcp/35442/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":280,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094","stored_metadata":{"remote_peer_id":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094","connection_id":280,"addr":"/ip4/211.91.254.188/tcp/35442/noise-ik/0x848f4e17d02ad1c906955924a8482ea7fb51c56de42002a253af02780a51bf69/handshake/0","origin":"Inbound","messaging_protocol":"V1","application_protocols":{"inner":[231,152]},"role":"Validator"}}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer/mod.rs:384"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620473Z","message":"[validator,Validator,21b4ef9b] Closed connection to peer: 9f9ae72a","data":{"connection_id":279,"connection_origin":"inbound","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620503Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 9f9ae72a round: 33 with error: UnexpectedResponseChannelCancel","data":{"error":"UnexpectedResponseChannelCancel","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1","round":33}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620524Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 9f9ae72a round: 32 with error: UnexpectedResponseChannelCancel","data":{"error":"UnexpectedResponseChannelCancel","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"9f9ae72a7539640b4ffc169e080a07b3276bca2527193f53232f27cbf12182d1","round":32}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620530Z","message":"[validator,Validator,21b4ef9b] Failed to connect to peer: d801e1b2 at address: /ip4/95.188.77.87/tcp/6181/noise-ik/0xcc98cf612f641136ff744bcfd0a3228219fa815e9269556b6d992c23236b462f/handshake/0; error: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/95.188.77.87/tcp/6181/noise-ik/0xcc98cf612f641136ff744bcfd0a3228219fa815e9269556b6d992c23236b462f/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"d801e1b2c79395df00ff26f63f60e2f11d541d18d6755831ec1a6537fd3ad6d3"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620537Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 5e841137 round: 33 with error: UnexpectedResponseChannelCancel","data":{"error":"UnexpectedResponseChannelCancel","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094","round":33}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/health_checker/mod.rs:381"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.620549Z","message":"[validator,Validator,21b4ef9b] Ping failed for peer: 5e841137 round: 32 with error: UnexpectedResponseChannelCancel","data":{"error":"UnexpectedResponseChannelCancel","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"5e8411375edd94cec20e313abaa9a4694f0af87df41942f74e8d7d432ffd3094","round":32}}
testnet-validator-1  | {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.692236Z","message":"[validator,Validator,21b4ef9b] Outbound connection failed for peer b4046267 at /ip4/92.255.176.91/tcp/6180/noise-ik/0xabac71d7faba0e2d26daffb8365b12706f6a09d24805b98259e197b1b30c9429/handshake/0: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/92.255.176.91/tcp/6180/noise-ik/0xabac71d7faba0e2d26daffb8365b12706f6a09d24805b98259e197b1b30c9429/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"b404626788b5ed7e6940200acfe6884f91cf636c62751bd12508d51bdb80e13a"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:04.692282Z","message":"[validator,Validator,21b4ef9b] Failed to connect to peer: b4046267 at address: /ip4/92.255.176.91/tcp/6180/noise-ik/0xabac71d7faba0e2d26daffb8365b12706f6a09d24805b98259e197b1b30c9429/handshake/0; error: Transport error: timeout elapsed","data":{"error":"Transport error: timeout elapsed","network_address":"/ip4/92.255.176.91/tcp/6180/noise-ik/0xabac71d7faba0e2d26daffb8365b12706f6a09d24805b98259e197b1b30c9429/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"b404626788b5ed7e6940200acfe6884f91cf636c62751bd12508d51bdb80e13a"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:728"},"thread_name":"aptos-data-client","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:05.005710Z","data":{"event":"priority_and_regular_peers","message":"Number of priority peers: 215. Number of regular peers: 1","name":"peer_states"}}
testnet-validator-1  | {"level":"INFO","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:805"},"thread_name":"aptos-data-client","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:05.170794Z","data":{"event":"aggregate_summary","message":"Global data summary: AdvertisedData { Highest epoch ending ledger info, epoch: Some(34). Highest synced ledger info, epoch: Some(35), version: Some(79518532). Lowest transaction version: Some(0), Lowest transaction output version: Some(0), Lowest states version: Some(534412) }, OptimalChunkSizes { epoch_chunk_size: 100, state_chunk_size: 2000, transaction_chunk_size: 2000, transaction_output_chunk_size: 2000 }","name":"peer_states"}}
testnet-validator-1  | {"level":"ERROR","source":{"package":"aptos_data_client","file":"state-sync/aptos-data-client/src/aptosnet/mod.rs:434"},"thread_name":"data-streaming-service","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:07.308771Z","data":{"error":"TimeoutWaitingForResponse(\"Rpc timed out\")","event":"response_error","name":"storage_service_response","peer":"Validator:afea3644","request_id":2837,"request_type":"get_transaction_outputs_with_proof_compressed"}}
testnet-validator-1  | {"level":"WARN","source":{"package":"network","file":"network/src/protocols/rpc/mod.rs:558"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:07.309091Z","message":"[validator,Validator,21b4ef9b] Error making outbound rpc request with request_id 44 to afea3644: Error: Rpc timed out\n\nStack backtrace:\n   0: network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:477:57\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      network::protocols::rpc::OutboundRpcs::handle_outbound_request::{{closure}}::{{closure}}\n             at ./aptos/network/src/protocols/rpc/mod.rs:490:37\n      <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n   1: <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      <futures_util::stream::futures_unordered::FuturesUnordered<Fut> as futures_core::stream::Stream>::poll_next\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/futures_unordered/mod.rs:514:17\n      futures_util::stream::stream::StreamExt::poll_next_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/mod.rs:1626:9\n   2: <futures_util::stream::stream::select_next_some::SelectNextSome<St> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/stream/stream/select_next_some.rs:34:36\n      <core::pin::Pin<P> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/future.rs:124:9\n      futures_util::future::future::FutureExt::poll_unpin\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/future/mod.rs:562:9\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n   3: core::ops::function::impls::<impl core::ops::function::FnMut<A> for &mut F>::call_mut\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:290:13\n      network::peer::Peer<TSocket>::start::{{closure}}::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/async_await/select_mod.rs:321:13\n      <futures_util::future::poll_fn::PollFn<F> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/futures-util-0.3.21/src/future/poll_fn.rs:56:9\n   4: network::peer::Peer<TSocket>::start::{{closure}}\n             at ./aptos/network/src/peer/mod.rs:254:13\n   5: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/future/mod.rs:91:19\n      tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n   6: tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n   7: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::LocalNotified<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:376:9\n      tokio::runtime::thread_pool::worker::Context::run_task::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:435:13\n      tokio::coop::with_budget::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:102:9\n      std::thread::local::LocalKey<T>::try_with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:445:16\n      std::thread::local::LocalKey<T>::with\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/local.rs:421:9\n   8: tokio::coop::with_budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:95:5\n      tokio::coop::budget\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/coop.rs:72:5\n      tokio::runtime::thread_pool::worker::Context::run_task\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:434:9\n   9: tokio::runtime::thread_pool::worker::Context::run\n  10: tokio::runtime::thread_pool::worker::run::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:386:17\n      tokio::macros::scoped_tls::ScopedKey<T>::set\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/macros/scoped_tls.rs:61:9\n  11: tokio::runtime::thread_pool::worker::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:383:5\n  12: tokio::runtime::thread_pool::worker::Launch::launch::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/thread_pool/worker.rs:362:45\n      <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/task.rs:42:21\n  13: tokio::runtime::task::core::CoreStage<T>::poll::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:165:17\n      tokio::loom::std::unsafe_cell::UnsafeCell<T>::with_mut\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/loom/std/unsafe_cell.rs:14:9\n      tokio::runtime::task::core::CoreStage<T>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/core.rs:155:13\n      tokio::runtime::task::harness::poll_future::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:470:19\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      tokio::runtime::task::harness::poll_future\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:458:18\n      tokio::runtime::task::harness::Harness<T,S>::poll_inner\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:104:27\n      tokio::runtime::task::harness::Harness<T,S>::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/harness.rs:57:15\n  14: tokio::runtime::task::raw::RawTask::poll\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/raw.rs:84:18\n      tokio::runtime::task::UnownedTask<S>::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/task/mod.rs:413:9\n      tokio::runtime::blocking::pool::Task::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:91:9\n      tokio::runtime::blocking::pool::Inner::run\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:325:17\n  15: tokio::runtime::blocking::pool::Spawner::spawn_thread::{{closure}}\n             at ./usr/local/cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.19.2/src/runtime/blocking/pool.rs:300:13\n      std::sys_common::backtrace::__rust_begin_short_backtrace\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys_common/backtrace.rs:122:18\n  16: std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:505:17\n      <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/panic/unwind_safe.rs:271:9\n      std::panicking::try::do_call\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:492:40\n      std::panicking::try\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panicking.rs:456:19\n      std::panic::catch_unwind\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/panic.rs:137:14\n      std::thread::Builder::spawn_unchecked_::{{closure}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/thread/mod.rs:504:30\n      core::ops::function::FnOnce::call_once{{vtable.shim}}\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/core/src/ops/function.rs:248:5\n  17: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/alloc/src/boxed.rs:1951:9\n      std::sys::unix::thread::Thread::new::thread_start\n             at ./rustc/4b91a6ea7258a947e59c6522cd5898e7c0a6a88f/library/std/src/sys/unix/thread.rs:108:17\n  18: start_thread\n             at ./build/glibc-fWwxX8/glibc-2.28/nptl/pthread_create.c:486:8\n  19: clone\n             at ./build/glibc-fWwxX8/glibc-2.28/misc/../sysdeps/unix/sysv/linux/x86_64/clone.S:95","data":{"network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"afea364425825255938bbfdb1704f06ec56d5c914833208b96fc47b94aff1671"}}
testnet-validator-1  | {"level":"ERROR","source":{"package":"data_streaming_service","file":"state-sync/state-sync-v2/data-streaming-service/src/data_stream.rs:413"},"thread_name":"data-streaming-service","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:07.406818Z","data":{"error":{"AptosDataClientError":"Timed out waiting for a response: Rpc timed out"},"event":"error","message":"Encountered a data client error!","name":"received_data_response","stream_id":2}}
testnet-validator-1  | {"level":"INFO","source":{"package":"network","file":"network/src/peer_manager/mod.rs:359"},"thread_name":"network-Validator","hostname":"8de8354f765c","timestamp":"2022-09-01T11:25:07.485882Z","message":"[validator,Validator,21b4ef9b] New connection established: [28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186,/ip4/5.161.137.197/tcp/36574/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0,inbound,V1,ProtocolIdSet(BitVec { inner: [231, 152] }),validator]","data":{"connection_id":298,"connection_origin":"inbound","network_address":"/ip4/5.161.137.197/tcp/36574/noise-ik/0xe26f5cdc2b89545b3481c055fac34781b585b4fa5b1d93bbf2865f7cb20b130c/handshake/0","network_context":{"role":"validator","network_id":"Validator","peer_id":"21b4ef9b8512ec0b801e78942580f04e10d3fc44148d0d6acbba0dd21cf1ce42"},"remote_peer":"28dde00b49a2c5ec8b46a1fbd855afd0fac64ea347a3dffc803ad672e7810186"}}

Hello Aptos Team!

My metrics tell me latest epoch 21

curl 80.211.33.21:9101/metrics 2> /dev/null | grep aptos_state_sync_epoch_state_gauges | awk -F',' '{print $1}' | awk -F'=' '{print $2}' | tr -d \" | sort -hu

But all history epoch get output:

curl https://aptos-analytics-data-it3.s3.us-west-2.amazonaws.com/validator_epoch_history.json | grep -B 5 0xfecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6

    {
        "epoch": 28,
        "failed_proposal_count": null,
        "max_potential_reward": 1598100000,
        "rewards_earned": 0,
        "successful_proposal_count": null,
        "validator_addr": "0xfecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6",
        "voting_power": 100000000000000
    },
    {
        "epoch": 29,
        "failed_proposal_count": null,
        "max_potential_reward": 1598100000,
        "rewards_earned": 0,
        "successful_proposal_count": null,
        "validator_addr": "0xfecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6",
        "voting_power": 100000000000000
    },
    {
        "epoch": 30,
        "failed_proposal_count": null,
        "max_potential_reward": 1598100000,
        "rewards_earned": 0,
        "successful_proposal_count": null,
        "validator_addr": "0xfecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6",
        "voting_power": 100000000000000
    },
    {
        "epoch": 31,
        "failed_proposal_count": null,
        "max_potential_reward": 1598100000,
        "rewards_earned": 0,
        "successful_proposal_count": null,
        "validator_addr": "0xfecae3412581c6d781196e1b3120b28bfb8463845bc1e508db43436844bf40d6",
        "voting_power": 100000000000000
    },

My node was running fine from the beginning, showing 100% liveness. But at some stage it lost synchronisation and now it is way behind. Still trying to catch up. Now it shows 0 liveness and 0 proposals (which could be expected because it's not synced)
Why it's in the down node list and also liveness is 0?

Petra owner address: 0x2f9baa63446d2894d84fdf3dfaff1a7de69cf47cdea0b518e05e33076ddcd377
Petra voter address: 0x2d3ed4b55a7c401f67a9a0d9fb14c0f327c76ac55b7023f79723d0670453580c
Operator address: 0x88824a786f30546624c57a60dc15cedea6e0945aba7005eafc665f5f9968fb63

KEYS in the Operator.yaml:
operator_account_address: 88824a786f30546624c57a60dc15cedea6e0945aba7005eafc665f5f9968fb63
operator_account_public_key: "0x4244211d19856ccdb47fd93986c7c0de9201ef7bf9b483ff661fc79dee6fe4c5"
consensus_public_key: "0xa9ffdff1a40ce4f9c2dab00f5c84ece4a7fd1f09dcc40316526388be35b77d07b9ecc95ad9e3141acafdb18c4c6f2478"
consensus_proof_of_possession: "0x8f2d8a3cca3414c72d56098be7ea5e036f6d930df34a2ac9d640a4c96dd78ffc8bfa9fc161ab303b8f36d3d858429bf90da8fc8429b58f61a326bbc8d36a0ae564ad6b5f4e57f0cea8a66cbf1908ce5************************
validator_network_public_key: "0xcd4d4ee22e5f4e0d1895f945ca0aeb16b369f5ad0e486b042a8509c406585d5b"

KEYS in validator-identity.yaml:
account_address: 0x2f9baa63446d2894d84fdf3dfaff1a7de69cf47cdea0b518e05e33076ddcd377
account_private_key: "0xc407f4d*******************
consensus_private_key: "0x2dd079*******************
network_private_key: "0x60ef10e21**********************

Running at commit 4219d75c57115805f0faf0ad07c17becf6580202

Stopped at night so now catching up:
aptos_state_sync_version{type="applied_transaction_outputs"} 30812965
aptos_state_sync_version{type="executed_transactions"} 30614965
aptos_state_sync_version{type="synced"} 30812965
aptos_state_sync_version{type="synced_epoch"} 20

PEERS. IP and DNS have just changed so probably still propagating hence 0 incoming:
aptos_connections{direction="inbound",network_id="Validator",peer_id="2f9baa63",role_type="validator"} 0
aptos_connections{direction="outbound",network_id="Validator",peer_id="2f9baa63",role_type="validator"} 183

Yesterday it was showing connected to APTOS node but today it's not. Could be related to the network issues at the moment?
"..sherry — Today at 10:14
@aIt3 If you see the network paused, it's the same issue again. we're running more test to root cause the issue, this might happen several times, sorry about the noise"

:~# curl 127.0.0.1:9101/metrics 2> /dev/null | grep "aptos_network_peer_connected{.remote_peer_id="".}"

Thanks.

Fullnode running docker tag: aptoslab/validator:testnet_4219d75c57115805f0faf0ad07c17becf6580202

It seems that after restarting, the synchronization is slowly turned on, but there are still some errors

running log:

fullnode.log

This is my owner wallet: 0xb41568279f81fccc440bdbb807f68eb0cbdd494c74de5cccf545b0a47d890e93 .
My validator block synchronization is unable to keep up with the speed of block generation.

After 12 hours, node was synchronized only 65992300. I have run on it latest version. And fullnode is in same situation.


Here is my validator log attachment.
log.log

Hi all,

I installed my node yesterday morning before 12am UTC and in about a couple of hours it started to produce all the necessary stuff like
aptos_consensus_current_round / aptos_consensus_proposals_count but i can see the proposals count barely increasing.

Connected to the Aptos node alright and i got good amount of inbound / outbound connections

And my stake keeps increasing!

Though at the moment the [AIT-3 Validator Status] page shows me 0% rewards

And i just wonder what could be the issue and a possible fix?

Peace.

Could you plz check my log
panic.log

OWNER ADDRESS 0x9eaa6c899597ca4d691cf453a1b0b17b534ed8fa91eac78a9e9bc0f457ee8cba

REWARDS PERFORMANCE 0 !!!!
aptos_consensus_current_round 1
aptos_consensus_proposals_count 0
Number not increasing !!!

aptos_connections{direction="inbound",network_id="Validator",peer_id="9eaa6c89",role_type="validator"} 29
aptos_connections{direction="outbound",network_id="Validator",peer_id="9eaa6c89",role_type="validator"} 166

root@vmi847187:~# curl 127.0.0.1:9101/metrics 2> /dev/null | grep "aptos_network_peer_connected{.remote_peer_id="f326fd30".}"
aptos_network_peer_connected{network_id="Validator",peer_id="9eaa6c89",remote_peer_id="f326fd30",role_type="validator"} 1

aptos_state_sync_version{type="applied_transaction_outputs"} 44965609
aptos_state_sync_version{type="executed_transactions"} 41414921
aptos_state_sync_version{type="synced"} 44965609
aptos_state_sync_version{type="synced_epoch"} 26

synchronized

StakePool not increasing

Owner
0xf2e802753bbaa2b3e0861180dfd2dd3fe8dc9bb0d49e0094376f6d565bab9884
This address is in the file you sent prevously (https://gist.github.com/sherry-x/f45f0f4e3f65f6eb5ab75ded339ac372) but I have no "The validator is not in the validator set. Address not in set" errors and validator-identity.yaml is correct.

aptos_consensus_current_round sometimes 0, sometimes after restart >0 (7318 now)
aptos_consensus_proposals_count always 0

aptos_state_sync_version{type="applied_transaction_outputs"} 53916947
stops firstly, then other metrics stop increasing sometimes, then I restart it.

Log.txt

Hello,
It seems like "locked until" is different between wallet and https://explorer.devnet.aptos.dev/proposals/staking

This is logs.
{"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:31.572002Z","message":"[full_node,Public,8e389e1b] Failed to connect to peer: 26a5f68e at address: /ip4/34.142.59.8/tcp/6182/noise-ik/0x5b038e37dfe74968e758191f609a5ab4dfb247aabef52720164bd9acbcf65934/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/34.142.59.8/tcp/6182/noise-ik/0x5b038e37dfe74968e758191f609a5ab4dfb247aabef52720164bd9acbcf65934/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"26a5f68e39a2543aef7ceaec7dcd047510d856b495f95979879bd278e4725e07"}} {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:31.607904Z","message":"[full_node,Public,8e389e1b] Outbound connection failed for peer 802f9bbd at /dns/node.it3.aptos.scw.testnet.kiln.fi/tcp/6182/noise-ik/0x075b7e96dbc18c956f0288611f409342b731c93242ec5f7150e87de6cc0c2e33/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/dns/node.it3.aptos.scw.testnet.kiln.fi/tcp/6182/noise-ik/0x075b7e96dbc18c956f0288611f409342b731c93242ec5f7150e87de6cc0c2e33/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"802f9bbd78574afe79289edf5adcce044ab23535dcd8e79289abf2cb492a3c57"}} {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:31.607953Z","message":"[full_node,Public,8e389e1b] Failed to connect to peer: 802f9bbd at address: /dns/node.it3.aptos.scw.testnet.kiln.fi/tcp/6182/noise-ik/0x075b7e96dbc18c956f0288611f409342b731c93242ec5f7150e87de6cc0c2e33/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/dns/node.it3.aptos.scw.testnet.kiln.fi/tcp/6182/noise-ik/0x075b7e96dbc18c956f0288611f409342b731c93242ec5f7150e87de6cc0c2e33/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"802f9bbd78574afe79289edf5adcce044ab23535dcd8e79289abf2cb492a3c57"}} {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:31.677229Z","message":"[full_node,Public,8e389e1b] Outbound connection failed for peer c54fc020 at /dns/aptos-vfn.lefey.ru/tcp/6182/noise-ik/0x3b2e63504b8b307e7385130d7f6fa4dbcbedd34bf716204f3e5f4996bb529449/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/dns/aptos-vfn.lefey.ru/tcp/6182/noise-ik/0x3b2e63504b8b307e7385130d7f6fa4dbcbedd34bf716204f3e5f4996bb529449/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"c54fc020dbf12a2f04c184e62ffd53f64a1c82f9d2bd6f4e074bed4f512148c4"}} {"level":"INFO","source":{"package":"network","file":"network/src/connectivity_manager/mod.rs:869"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:31.677285Z","message":"[full_node,Public,8e389e1b] Failed to connect to peer: c54fc020 at address: /dns/aptos-vfn.lefey.ru/tcp/6182/noise-ik/0x3b2e63504b8b307e7385130d7f6fa4dbcbedd34bf716204f3e5f4996bb529449/handshake/0; error: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/dns/aptos-vfn.lefey.ru/tcp/6182/noise-ik/0x3b2e63504b8b307e7385130d7f6fa4dbcbedd34bf716204f3e5f4996bb529449/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"c54fc020dbf12a2f04c184e62ffd53f64a1c82f9d2bd6f4e074bed4f512148c4"}} {"level":"ERROR","source":{"package":"network","file":"network/src/peer_manager/transport.rs:258"},"thread_name":"network-Public","hostname":"f3c1f136f3d6","timestamp":"2022-09-01T00:24:36.759602Z","message":"[full_node,Public,8e389e1b] Outbound connection failed for peer b7aa84d8 at /ip4/104.197.116.46/tcp/6182/noise-ik/0x881dfe88146b45afa1162dcd0ba84f8717a7e28df5733f6e947257a0a42b566b/handshake/0: Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","data":{"error":"Transport error: noise client: error reading server handshake response message, server probably rejected our handshake message: unexpected end of file","network_address":"/ip4/104.197.116.46/tcp/6182/noise-ik/0x881dfe88146b45afa1162dcd0ba84f8717a7e28df5733f6e947257a0a42b566b/handshake/0","network_context":{"role":"full_node","network_id":"Public","peer_id":"8e389e1bdc0bb4f783403ffcaa793153de8f88e36dee511a11d7dce8be8c0714"},"remote_peer":"b7aa84d8be549026574559735403a79b001ca2c3be54d405b8ec7c4df79c1b15"}}

The fullnode sync_version is always in height 44613609
`root@instance-2:~/aptos-node/testnet3# curl 127.0.0.1:9101/metrics 2> /dev/null | grep "sync_version"

HELP aptos_state_sync_version The versions processed by the storage synchronizer operations

TYPE aptos_state_sync_version gauge

aptos_state_sync_version{type="applied_transaction_outputs"} 44603609
aptos_state_sync_version{type="executed_transactions"} 44423609
aptos_state_sync_version{type="synced"} 44603609
aptos_state_sync_version{type="synced_epoch"} 26
root@instance-2:~/aptos-node/testnet3# curl 127.0.0.1:9101/metrics 2> /dev/null | grep "sync_version"

HELP aptos_state_sync_version The versions processed by the storage synchronizer operations

TYPE aptos_state_sync_version gauge

aptos_state_sync_version{type="applied_transaction_outputs"} 44613609
aptos_state_sync_version{type="executed_transactions"} 44423609
aptos_state_sync_version{type="synced"} 44611609
aptos_state_sync_version{type="synced_epoch"} 26`

help me please