dmdcoin / diamond-node Goto Github PK
View Code? Open in Web Editor NEWbit.diamonds node software for network version 4
License: GNU General Public License v3.0
diamond-node's People
Contributors
Watchers
diamond-node's Issues
Prometheus logging can result in a deadlock
happened on Hbbft2:
2023-05-14 13:17:43 removing 0 reserved peers, because they are neither a pending validator nor a current validator.
2023-05-14 13:17:43 POSDAO epoch changed from 5751 to 5752.
2023-05-14 13:17:43 report new epoch: 5752
2023-05-14 13:17:56 1 deadlock(s) detected
2023-05-14 13:17:56 Deadlock #0
2023-05-14 13:17:56 Thread Id 140085339076352
2023-05-14 13:17:56 0: 0x55be3c7a3656 - backtrace::backtrace::libunwind::trace::he8f90d654b8dc5ea
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.56/src/backtrace/libunwind.rs:90:5
backtrace::backtrace::trace_unsynchronized::he1343eb1eec60890
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.56/src/backtrace/mod.rs:66:5
backtrace::backtrace::trace::ha23987a553685e3a
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.56/src/backtrace/mod.rs:53:14
backtrace::capture::Backtrace::create::h415567cce1e4ac38
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.56/src/capture.rs:176:9
1: 0x55be3c7a3583 - backtrace::capture::Backtrace::new::h16b94d5845ee5022
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.56/src/capture.rs:140:22
2: 0x55be3c78b0b9 - parking_lot_core::parking_lot::deadlock_impl::on_unpark::h2768f2c8fd347845
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:1179:32
3: 0x55be3b86e6b8 - parking_lot_core::parking_lot::deadlock::on_unpark::he364b6c75340f6a3
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:1112:9
parking_lot_core::parking_lot::park::{{closure}}::hef55225fa08bc95a
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:613:17
parking_lot_core::parking_lot::with_thread_data::h01cef5c602b00f0e
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:183:5
parking_lot_core::parking_lot::park::hbff0db6976c66144
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:576:5
parking_lot::raw_rwlock::RawRwLock::lock_common::hbe012f6602ff15f9
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:1110:17
parking_lot::raw_rwlock::RawRwLock::lock_shared_slow::hc30010f2017ce8cb
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:714:9
4: 0x55be3c176ad7 - <parking_lot::raw_rwlock::RawRwLock as lock_api::rwlock::RawRwLock>::lock_shared::hcc166ebfb197ce55
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:110:26
lock_api::rwlock::RwLock<R,T>::read::hfbb023e42636a8a5
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/lock_api-0.4.6/src/rwlock.rs:448:9
<ethcore::client::client::Client as ethcore::client::traits::BlockChainClient>::pruning_info::h5b36257a895e08fe
at /home/dmdnode/v0.8.6-rc1/diamond-node-git/crates/ethcore/src/client/client.rs:2798:29
5: 0x55be3c188f3c - <ethcore::client::client::Client as stats::PrometheusMetrics>::prometheus_metrics::h9eb2723d82cc59c1
at /home/dmdnode/v0.8.6-rc1/diamond-node-git/crates/ethcore/src/client/client.rs:3624:24
6: 0x55be3b9a6042 - diamond_node::metrics::handle_request::hae3a7c30a9af123c
at /home/dmdnode/v0.8.6-rc1/diamond-node-git/bin/oe/metrics.rs:53:13
diamond_node::metrics::start_prometheus_metrics::{{closure}}::{{closure}}::h6c9dfa62f925fdb3
at /home/dmdnode/v0.8.6-rc1/diamond-node-git/bin/oe/metrics.rs:108:17
<hyper::service::service::ServiceFnOk<F,ReqBody> as hyper::service::service::Service>::call::he59da67aac3a97da
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/service/service.rs:155:20
7: 0x55be3b99b2ab - <hyper::proto::h1::dispatch::Server<S> as hyper::proto::h1::dispatch::Dispatch>::recv_msg::h87817851a5449a69
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:431:31
8: 0x55be3bb454bd - hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read_head::hd647fefd711084ff
hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_read::h55f90ac45b8901b0
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:163:28
9: 0x55be3b9e75b9 - hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_loop::h23a2ec8dd8c18868
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:129:13
hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner::h22f06291f1e3355b
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:106:20
hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch::h33e0e63de1d9dbab
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:93:9
<hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T> as futures::future::Future>::poll::h310c433feb1b5afa
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/proto/h1/dispatch.rs:374:9
<futures::future::either::Either<A,B> as futures::future::Future>::poll::h634d73ffa93abc69
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/either.rs:35:37
futures::future::option::<impl futures::future::Future for core::option::Option<F>>::poll::h31b358d9eadb1d3c
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/option.rs:12:32
<hyper::server::conn::upgrades::UpgradeableConnection<I,S,E> as futures::future::Future>::poll::hd345d9f3420d51eb
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/server/conn.rs:948:23
10: 0x55be3b9e6248 - <hyper::server::conn::spawn_all::NewSvcTask<I,N,S,E,W> as futures::future::Future>::poll::h2ab98e29eea72558
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/hyper-0.12.35/src/server/conn.rs:888:32
11: 0x55be3c613de4 - <alloc::boxed::Box<F> as futures::future::Future>::poll::h894625b41fa8f2bd
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/mod.rs:113:13
futures::task_impl::Spawn<T>::poll_future_notify::{{closure}}::h93d92e96b2881a7a
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329:45
futures::task_impl::Spawn<T>::enter::{{closure}}::h5bebde7ebfe368df
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399:27
futures::task_impl::std::set::hea93e4d2353a0145
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/std/mod.rs:83:13
futures::task_impl::Spawn<T>::enter::h0143ad0126ebbd90
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399:9
futures::task_impl::Spawn<T>::poll_fn_notify::h185eec319dc08456
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:291:9
futures::task_impl::Spawn<T>::poll_future_notify::h2a95cfc7c5c5074c
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329:9
tokio_threadpool::task::Task::run::{{closure}}::h6e9446654b8f2809
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/task/mod.rs:145:17
core::ops::function::FnOnce::call_once::h94329b367c79520a
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:250:5
<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h48a5ce0f3b771954
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/panic/unwind_safe.rs:271:9
std::panicking::try::do_call::h31fdaa88d5ddd6bc
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
std::panicking::try::ha5afbd13d59f9472
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
std::panic::catch_unwind::h1dc64362ca43cec6
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
tokio_threadpool::task::Task::run::hf2592af154934a1b
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/task/mod.rs:130:19
tokio_threadpool::worker::Worker::run_task2::h37f47a2c6e34ea87
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:567:9
tokio_threadpool::worker::Worker::run_task::h1395086bcd164435
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:459:19
12: 0x55be3c61299d - tokio_threadpool::worker::Worker::try_run_owned_task::h88e88a8241a88ce6
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:390:17
tokio_threadpool::worker::Worker::try_run_task::h4d3c033cc0c18eca
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:297:12
tokio_threadpool::worker::Worker::run::hb58558875c254828
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:241:21
13: 0x55be3c5f954d - tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::{{closure}}::haf65d9a91a3265c7
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:390:29
tokio_timer::timer::handle::with_default::h8ff29d78e66c5a2f
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.13/src/timer/handle.rs:74:5
tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::{{closure}}::hc642dd644d817fae
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:382:25
tokio_timer::clock::clock::with_default::h15778afb3910f45b
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-timer-0.2.13/src/clock/clock.rs:125:5
tokio::runtime::threadpool::builder::Builder::build::{{closure}}::{{closure}}::hcca6c07048a98f22
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:381:21
tokio_reactor::with_default::h12973e649dffe57b
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-reactor-0.1.12/src/lib.rs:220:5
tokio::runtime::threadpool::builder::Builder::build::{{closure}}::ha76f4488573222df
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.1.22/src/runtime/threadpool/builder.rs:380:17
14: 0x55be3c60a2dc - tokio_threadpool::callback::Callback::call::h5c4044e51c4aa315
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/callback.rs:22:9
tokio_threadpool::worker::Worker::do_run::{{closure}}::{{closure}}::hdf2eeda86de91be9
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:127:21
tokio_executor::global::with_default::{{closure}}::h16490220ce3391a5
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.10/src/global.rs:221:9
std::thread::local::LocalKey<T>::try_with::h362cf44613c9e3b1
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/local.rs:446:16
std::thread::local::LocalKey<T>::with::hd0b3959126c34ae3
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/local.rs:422:9
tokio_executor::global::with_default::h37a6882753676a2a
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-executor-0.1.10/src/global.rs:190:5
tokio_threadpool::worker::Worker::do_run::{{closure}}::hf085a17262bdab46
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:125:13
std::thread::local::LocalKey<T>::try_with::h950fa03ac57b299f
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/local.rs:446:16
std::thread::local::LocalKey<T>::with::hb70fb4ab5e7e9da9
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/local.rs:422:9
tokio_threadpool::worker::Worker::do_run::h6cd755f3ee1320fb
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/worker/mod.rs:116:9
tokio_threadpool::pool::Pool::spawn_thread::{{closure}}::h3252b07619505c46
at /home/dmdnode/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.18/src/pool/mod.rs:345:21
std::sys_common::backtrace::__rust_begin_short_backtrace::h5dacdbf32b05d787
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys_common/backtrace.rs:121:18
15: 0x55be3c60b293 - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hc77fd360e874fd99
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/mod.rs:558:17
<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h2185beb7115eb0af
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/panic/unwind_safe.rs:271:9
std::panicking::try::do_call::hb3ddaf96778e8429
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:483:40
std::panicking::try::he141366359aeb8b1
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panicking.rs:447:19
std::panic::catch_unwind::h19935dc880e1c695
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/panic.rs:140:14
std::thread::Builder::spawn_unchecked_::{{closure}}::h861244ed4e7a379a
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/thread/mod.rs:557:30
core::ops::function::FnOnce::call_once{{vtable.shim}}::heed0ba115bab04eb
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/core/src/ops/function.rs:250:5
16: 0x55be3ca572f3 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h3205ec2d7fc232c5
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/alloc/src/boxed.rs:1988:9
<alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h3bb5daec8177f56b
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/alloc/src/boxed.rs:1988:9
std::sys::unix::thread::Thread::new::thread_start::had7b8061e306bb5c
at /rustc/9eb3afe9ebe9c7d2b84b71002d44f4a0edac95e0/library/std/src/sys/unix/thread.rs:108:17
17: 0x7f683a5b06db - start_thread
18: 0x7f6839d3761f - clone
19: 0x0 - <unknown>
2023-05-14 13:18:06 eth_accounts is deprecated and will be removed in future versions: Account management is being phased out see #9997 for alternatives.
2023-05-14 13:18:39 do_validator_engine_actions
provide hbbft data by via Prometheus Interface.
failed to resolve: could not find `EthabfunctionsContract` in `ethabi_derive` on some systems
does not appear on every system.
It's known that that problem appeared with the hardhat upgrade, but in this case, the contracts that cause the problem have not been updated.
tested with rust 1.59 and 1.64.
another Hint is how Cargo.lock is working
error[E0433]: failed to resolve: could not find `EthabfunctionsContract` in `ethabi_derive`
--> crates/concensus/miner/src/service_transaction_checker.rs:26:1
|
26 | / use_contract!(
27 | | service_transaction,
28 | | "res/contracts/service_transaction.json"
29 | | );
| |_^ could not find `EthabfunctionsContract` in `ethabi_derive`
|
= note: this error originates in the macro `use_contract` (in Nightly builds, run with -Z macro-backtrace for more info)
error: cannot find attribute `ethabi_contract_options` in this scope
--> crates/concensus/miner/src/service_transaction_checker.rs:26:1
|
26 | / use_contract!(
27 | | service_transaction,
28 | | "res/contracts/service_transaction.json"
29 | | );
| |_^
|
= note: this error originates in the macro `use_contract` (in Nightly builds, run with -Z macro-backtrace for more info)
error[E0433]: failed to resolve: could not find `functions` in `service_transaction`
--> crates/concensus/miner/src/service_transaction_checker.rs:121:52
|
121 | let (data, decoder) = service_transaction::functions::certified::call(sender);
| ^^^^^^^^^ could not find `functions` in `service_transaction`
check for EVM changes in shanghai update
- for relevant changes in contract development.
hbbft message memorial - unprocessable messages
messages that could not be stored in any epoch keep getting processed again and again.
i think this happens when a node receives a message that was from the staking epoch before the current one.
Candidate availability bonus point system
Defines a fair system without low overhead that improves the score of a validator candidate node, if a candidate validator nodes checks and the candidate becomes available.
Interactions:
Uses automatic peer management: DMDcoin/openethereum-3.x#5
Nodes might loose bonus points by failing DMDcoin/openethereum-3.x#75
Diamond Network (not defined yet)
hbbft automatic reserved peers management
see DMDcoin/openethereum-3.x#5
took over from openethereum-3.x, because by luck we have the same issue ID
Write random number fork block
The new Feature of writing random numbers in Header and System Call has been included into the Node Software.
(DMDcoin/openethereum-3.x#26)
To mimic the old behavior during Sync it is required to specify a block when the new behavior should trigger, so the software neither writes the header information nor does the system calls.
This happened on the alpha testnet around Block 288968:
http://explorer.uniq.diamonds/block/288968/transactions
getting lost coins from inactive validators
See whitepaper: 5.5 Reinsert pot:
... when validators are inactive for 10 years...
Implement as public available function that move all coins (those from validator candidates and those that have delegated coins on it.
Nonce problem with announce availability and setting IP address
The Nonce management does not work in the case where we have more operations that send transaction at about the same time.
Here in this example, we are announcing availability and we are announcing our own IP address.
One of those transaction gets replaced,
but the Node software does not to try to send this transaction again, because it already did send a transaction.
We did have Nodes that were marked as Unavailable, but they were running,
and we did have Nodes that should have written their IP address, but they did not.
Probably the Nonce problem was the root of this problem.
A Node restart solved that problem, because the Node had only to either announce availability, or announce the IP address, whatever transaction was dropped & replaced at the last boot.
Log Entry:
2023-04-11 16:18:25 UTC Worker Hbbft2 WARN consensus do_validator_engine_actions
2023-04-11 16:18:25 UTC Worker Hbbft2 INFO engine sending announce availability transaction
2023-04-11 16:18:25 UTC Worker Hbbft2 INFO consensus sending announce availability with nonce: 0
2023-04-11 16:18:25 UTC Worker Hbbft2 TRACE txqueue Checking service transaction checker contract from 0x9640…84ce
2023-04-11 16:18:25 UTC Worker Hbbft2 TRACE txqueue Checking service transaction checker contract from 0x9640…84ce
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue Service tx 0xeefaf483ec4fa37d5b63de03fb5956d068128db4b7f813387a7ffff678b3a83f below minimal gas price accepted
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xeefaf483ec4fa37d5b63de03fb5956d068128db4b7f813387a7ffff678b3a83f] priority: Local
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue importing pool status: LightStatus { mem_usage: 0, transaction_count: 0, senders: 0 }
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xeefaf483ec4fa37d5b63de03fb5956d068128db4b7f813387a7ffff678b3a83f] Added to the pool.
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xeefaf483ec4fa37d5b63de03fb5956d068128db4b7f813387a7ffff678b3a83f] Sender: 0x9640…84ce, nonce: 0, gasPrice: 0, gas: 1000000, value: 0, dataLen: 68))
2023-04-11 16:18:25 UTC Worker Hbbft2 WARN engine checking if internet address needs to be updated.
2023-04-11 16:18:25 UTC Worker Hbbft2 WARN engine current Endpoint: 192.168.0.130:30303
2023-04-11 16:18:25 UTC Worker Hbbft2 WARN engine stored validator address0.0.0.1:0
2023-04-11 16:18:25 UTC Worker Hbbft2 INFO consensus set_validator_internet_address: ip: 192.168.0.130:30303 nonce: 0
2023-04-11 16:18:25 UTC Worker Hbbft2 TRACE txqueue Checking service transaction checker contract from 0x9640…84ce
2023-04-11 16:18:25 UTC Worker Hbbft2 TRACE txqueue Checking service transaction checker contract from 0x9640…84ce
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue Service tx 0xf7896b843fe6b66a2f6c0a403bd1e45009d6f96a53799880c8ae6b7987d696c7 below minimal gas price accepted
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xf7896b843fe6b66a2f6c0a403bd1e45009d6f96a53799880c8ae6b7987d696c7] priority: Local
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue importing pool status: LightStatus { mem_usage: 72, transaction_count: 1, senders: 1 }
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xf7896b843fe6b66a2f6c0a403bd1e45009d6f96a53799880c8ae6b7987d696c7] Added to the pool.
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xf7896b843fe6b66a2f6c0a403bd1e45009d6f96a53799880c8ae6b7987d696c7] Sender: 0x9640…84ce, nonce: 0, gasPrice: 0, gas: 100000, value: 0, dataLen: 68))
2023-04-11 16:18:25 UTC Worker Hbbft2 DEBUG txqueue [0xeefaf483ec4fa37d5b63de03fb5956d068128db4b7f813387a7ffff678b3a83f] Dropped. Replaced by [0xf7896b843fe6b66a2f6c0a403bd1e45009d6f96a53799880c8ae6b7987d696c7]
2
deadlock on locking HBBFT State after receiving Agreements
The reason for not producing blocks is an infinite lock on the hbbft worker thread that tries to write the hbbft state:
- State Write:
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::start_hbbft_epoch - State Write
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::process_hb_message - State Write:
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::replay_cached_messages
This bug is new and might be connected to the integration of either peer management or hbbft message tracking
Woker HBBFT 0 (This is the Engine Main Loop)
syscall (@syscall:12)
parking_lot_core::thread_parker::imp::ThreadParker::futex_wait (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:112)
<parking_lot_core::thread_parker::imp::ThreadParker as parking_lot_core::thread_parker::ThreadParkerT>::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:66)
parking_lot_core::parking_lot::park::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:611)
parking_lot_core::parking_lot::with_thread_data (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:183)
parking_lot_core::parking_lot::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:576)
parking_lot::raw_rwlock::RawRwLock::lock_common (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:1110)
parking_lot::raw_rwlock::RawRwLock::lock_exclusive_slow (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:628)
<parking_lot::raw_rwlock::RawRwLock as lock_api::rwlock::RawRwLock>::lock_exclusive (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:74)
lock_api::rwlock::RwLock<R,T>::write (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/lock_api-0.4.3/src/rwlock.rs:461)
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::start_hbbft_epoch (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:724)
<ethcore::engines::hbbft::hbbft_engine::TransitionHandler as ethcore_io::IoHandler<()>>::timeout (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:210)
ethcore_io::worker::Worker::do_work (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:135)
ethcore_io::worker::Worker::new::{{closure}}::{{closure}} (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:98)
futures::future::loop_fn::loop_fn (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/loop_fn.rs:79)
ethcore_io::worker::Worker::new::{{closure}} (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:86)
std::sys_common::backtrace::__rust_begin_short_backtrace (@std::sys_common::backtrace::__rust_begin_short_backtrace:17)
std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}} (@core::ops::function::FnOnce::call_once{{vtable.shim}}:70)
<core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once (@core::ops::function::FnOnce::call_once{{vtable.shim}}:58)
std::panicking::try::do_call (@core::ops::function::FnOnce::call_once{{vtable.shim}}:58)
a second thread that tries to write on the state is
Worker Client 1
syscall (@syscall:12)
parking_lot_core::thread_parker::imp::ThreadParker::futex_wait (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:112)
<parking_lot_core::thread_parker::imp::ThreadParker as parking_lot_core::thread_parker::ThreadParkerT>::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:66)
parking_lot_core::parking_lot::park::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:611)
parking_lot_core::parking_lot::with_thread_data (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:183)
parking_lot_core::parking_lot::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:576)
parking_lot::raw_rwlock::RawRwLock::lock_common (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:1110)
parking_lot::raw_rwlock::RawRwLock::lock_exclusive_slow (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:628)
<parking_lot::raw_rwlock::RawRwLock as lock_api::rwlock::RawRwLock>::lock_exclusive (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:74)
lock_api::rwlock::RwLock<R,T>::write (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/lock_api-0.4.3/src/rwlock.rs:461)
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::process_hb_message (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:524)
<ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT as ethcore::engines::Engine<ethcore::machine::impls::EthereumMachine>>::handle_message (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:1305)
<ethcore::client::client::Client as ethcore::client::traits::IoClient>::queue_consensus_message::{{closure}} (/home/sn/dmd/diamond-node/crates/ethcore/src/client/client.rs:2976)
ethcore::client::client::IoChannelQueue::queue::{{closure}} (/home/sn/dmd/diamond-node/crates/ethcore/src/client/client.rs:3520)
<ethcore_service::service::ClientIoHandler as ethcore_io::IoHandler<ethcore::client::io_message::ClientIoMessage>>::message (/home/sn/dmd/diamond-node/crates/ethcore/service/src/service.rs:229)
ethcore_io::worker::Worker::do_work (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:139)
ethcore_io::worker::Worker::new::{{closure}}::{{closure}} (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:98)
<futures::future::loop_fn::LoopFn<A,F> as futures::future::Future>::poll (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/loop_fn.rs:95)
futures::task_impl::Spawn<T>::poll_future_notify::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329)
futures::task_impl::Spawn<T>::enter::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:399)
Worker Client 2
syscall (@syscall:12)
parking_lot_core::thread_parker::imp::ThreadParker::futex_wait (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:112)
<parking_lot_core::thread_parker::imp::ThreadParker as parking_lot_core::thread_parker::ThreadParkerT>::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/thread_parker/linux.rs:66)
parking_lot_core::parking_lot::park::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:611)
parking_lot_core::parking_lot::with_thread_data (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:183)
parking_lot_core::parking_lot::park (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot_core-0.8.3/src/parking_lot.rs:576)
parking_lot::raw_rwlock::RawRwLock::lock_common (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:1110)
parking_lot::raw_rwlock::RawRwLock::lock_exclusive_slow (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:628)
<parking_lot::raw_rwlock::RawRwLock as lock_api::rwlock::RawRwLock>::lock_exclusive (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/parking_lot-0.11.1/src/raw_rwlock.rs:74)
lock_api::rwlock::RwLock<R,T>::write (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/lock_api-0.4.3/src/rwlock.rs:461)
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::join_hbbft_epoch (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:708)
ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT::process_hb_message (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:546)
<ethcore::engines::hbbft::hbbft_engine::HoneyBadgerBFT as ethcore::engines::Engine<ethcore::machine::impls::EthereumMachine>>::handle_message (/home/sn/dmd/diamond-node/crates/ethcore/src/engines/hbbft/hbbft_engine.rs:1305)
<ethcore::client::client::Client as ethcore::client::traits::IoClient>::queue_consensus_message::{{closure}} (/home/sn/dmd/diamond-node/crates/ethcore/src/client/client.rs:2976)
ethcore::client::client::IoChannelQueue::queue::{{closure}} (/home/sn/dmd/diamond-node/crates/ethcore/src/client/client.rs:3520)
<ethcore_service::service::ClientIoHandler as ethcore_io::IoHandler<ethcore::client::io_message::ClientIoMessage>>::message (/home/sn/dmd/diamond-node/crates/ethcore/service/src/service.rs:229)
ethcore_io::worker::Worker::do_work (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:139)
ethcore_io::worker::Worker::new::{{closure}}::{{closure}} (/home/sn/dmd/diamond-node/crates/runtime/io/src/worker.rs:98)
<futures::future::loop_fn::LoopFn<A,F> as futures::future::Future>::poll (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/future/loop_fn.rs:95)
futures::task_impl::Spawn<T>::poll_future_notify::{{closure}} (/home/sn/.cargo/registry/src/github.com-1ecc6299db9ec823/futures-0.1.29/src/task_impl/mod.rs:329)
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.