jimmychu0807 / substrate-offchain-worker-demo Goto Github PK

This is the src of price-fetch using simple-json.

The problem is if I compile as is, it returns:

error[E0433]: failed to resolve: use of undeclared type or module `alloc`
 --> /home/jimmychu/.cargo/git/checkouts/simple-json-311b636820194414/2b52149/src/impls.rs:3:5
  |
3 | use alloc::vec::Vec;
  |     ^^^^^ use of undeclared type or module `alloc`

error[E0433]: failed to resolve: use of undeclared type or module `std`
 --> /home/jimmychu/.cargo/git/checkouts/simple-json-311b636820194414/2b52149/src/json.rs:6:5
  |
6 | use std::convert::TryInto;
  |     ^^^ use of undeclared type or module `std`

error[E0433]: failed to resolve: use of undeclared type or module `Vec`
   --> /home/jimmychu/.cargo/git/checkouts/simple-json-311b636820194414/2b52149/src/json.rs:105:29
    |
105 |             Either::B(_) => Vec::new(),
    |                             ^^^ use of undeclared type or module `Vec`

error[E0433]: failed to resolve: use of undeclared type or module `Vec`
   --> /home/jimmychu/.cargo/git/checkouts/simple-json-311b636820194414/2b52149/src/json.rs:264:29
    |
264 |             Either::B(_) => Vec::new(),
    |                             ^^^ use of undeclared type or module `Vec`
...

(btw, the 2nd error is using std::convert::TryInto, I guess it is not totally no_std. Maybe should update with core::convert::TryInto).

I tried in Cargo.toml commenting out default_features=false, then it returns

error: duplicate lang item in crate `std` (which `simple_json` depends on): `panic_impl`.
  |
  = note: first defined in crate `sr_io` (which `offchain_node_runtime` depends on).

error: duplicate lang item in crate `std` (which `simple_json` depends on): `oom`.
  |
  = note: first defined in crate `sr_io` (which `offchain_node_runtime` depends on).

So this seems using std.

My problem: I don't know how to make price_fetch.rs compile with simple-json in no_std mode. Will be great if you can shed some light. @shawntabrizi @JoshOrndorff 🙏

My current workaround is conditionally compiling for price_fetch.rs as in still-good branch. (Search for #[cfg(feature = "std")])

There are two warnings about the deprecated ValidateUnsigned It might be a fun exercise to do as the warning suggests and try to update to SignedExtension instead.

warning: use of deprecated item 'sp_api_hidden_includes_construct_runtime::hidden_include::sp_runtime::traits::ValidateUnsigned': Use SignedExtensions instead.
   --> runtime/src/price_fetch.rs:345:16
    |
345 | impl<T: Trait> support::unsigned::ValidateUnsigned for Module<T> {
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |
    = note: `#[warn(deprecated)]` on by default

warning: use of deprecated item 'sp_api_hidden_includes_construct_runtime::hidden_include::sp_runtime::traits::ValidateUnsigned::Call': Use SignedExtensions instead.
   --> runtime/src/price_fetch.rs:348:31
    |
348 |   fn validate_unsigned(call: &Self::Call) -> TransactionValidity {
    |                               ^^^^^^^^^^

Hi Jimmy,
I m experimenting with substrate and OCWs and run into that error below. I looked for the library also manually, but did not find it. Can you help out or point me to the right repo, I would test and create a PR then.

Thx!

➜ substrate git:(master) cargo run --release -- --dev --tmp

Updating git repository `https://github.com/jimmychu0807/simple-json`

error: failed to get simple-json as a dependency of package pallet-price-fetch v1.0.0 (/home/w/polka/substrate-offchain-pricefetch/substrate/pallets/price-fetch)

Caused by:
failed to load source for dependency simple-json

Caused by:
Unable to update https://github.com/jimmychu0807/simple-json#e5e51a27

Caused by:
failed to fetch into: /home/w/.cargo/git/db/simple-json-aa2c7529b6e8fe90

Caused by:
process didn't exit successfully: git fetch --force --update-head-ok 'https://github.com/jimmychu0807/simple-json' 'refs/heads/master:refs/remotes/origin/master' 'HEAD:refs/remotes/origin/HEAD' (exit code: 128)
--- stderr
remote: Repository not found.
fatal: repository 'https://github.com/jimmychu0807/simple-json/' not found

Try update to more updated version of substrate and see if it fix the issue

The compiler gives unused import warnings about several traits.

warning: unused imports: `offchain::SignAndSubmitTransaction`, `offchain::SubmitSignedTransaction`
  --> runtime/src/price_fetch.rs:16:3
   |
16 |   offchain::SubmitSignedTransaction,
   |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
17 |   offchain::SubmitUnsignedTransaction,
18 |   offchain::SignAndSubmitTransaction };
   |   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: `#[warn(unused_imports)]` on by default

warning: unused import: `num_traits::float::FloatCore`
  --> runtime/src/price_fetch.rs:25:5
   |
25 | use num_traits::float::FloatCore;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

However all of these traits are actually used. Removing any one of them will create a compile error. At first this seemed related to rust-lang/rust#45268 but that issue is always about using modules, so I'm not sure it's the same.

The readme is still from Substrate Package. It could explain a bit about what this node does and how to use it.

Refer to the src: https://github.com/jimmychu0807/substrate-offchain-pricefetch/blob/still-good/node/runtime/src/offchaincb.rs#L120-L128

The example is based off from Ben's callback example. If there is an offchain request, the offchain worker will send both a signed and unsigned transaction back onchain.

I run the node in dev mode. I expect this makes Alice, Bob,... keys accessible in the runtime.

Questions

1. For signed tx, I first used polkadot-JS app and made a sudo call to add_authority() with Alice key. But in calling offchain_signed(), authority_id() function still return None. I suspect the local_keys are empty. How could I add Alice key in the local_keys?

As a side question, can I print out what local_keys is in the terminal?
@gnunicorn 🙏

2. For unsigned tx, I implemented validate_unsigned(), basically copying the core from im-online/src/lib.rs, with the returning ValidTransaction provides parameter includes nothing versus to (current_session, authority_id).encode() in im-online/src/lib.rs. It still returns Extrinsic invalid message for me.
   @tomusdrw 🙏

Detail output from terminal

2019-11-26 13:49:10 Idle (0 peers), best: #6 (0xb42f…62b7), finalized #5 (0xa5ea…3613), ⬇ 0 ⬆ 0
2019-11-26 13:49:12 Starting consensus session on top of parent 0xb42f5068b947b861666759c38ce87e568885b82af49cbf5519dae6ccd45b62b7
pinging
2019-11-26 13:49:12 Prepared block for proposing at 7 [hash: 0xaf0d152aae662639af94ae68cea39264d53f27ce1fa7dc930f80f566e72b45c5; parent_hash: 0xb42f…62b7; extrinsics: [0x122a…d4fd, 0x0f21…d0d6]]
2019-11-26 13:49:12 Pre-sealed block for proposal at 7. Hash now 0x7280ccf5a62ade19131fc81b311d97acc7b93dec1f6d84254166b3a13b449db5, previously 0xaf0d152aae662639af94ae68cea39264d53f27ce1fa7dc930f80f566e72b45c5.
pinging
2019-11-26 13:49:12 Imported #7 (0x7280…9db5)
offchain_worker
offchain_unsigned
2019-11-26 13:49:12 Submitting to the pool: 0308020b (isSigned: None)
authority not found
2019-11-26 13:49:12 Extrinsic invalid: 0x6bb01bd687515f3b69884c2693d33629b2e332aa449e1c7f352fddc7e2204474

• Refactored your pallet and separate the node out
• Switch ValidateUnsigned over to SignedExtension
• Write tests
• Make a 3-node setup: 1 node fetches from endpoint-X, another from endpoint-Y, and the third node as a medianizer.
• Gav: a price feed for kusama
• Gav: use transactions signed with the session key. Should not use "standard" signed transactions as then validators would need a hot account. (this is technically an UnsignedTransaction, but with a payload signed with session key)
• Gav: only allow validators to submit price
• Gav: only reward validators for the price when they're close to the median; slash validators when they're far off.
• blockFetch duration / remote endpoints all stored in storage and be updated on the fly, and managed via governance.

I tried the price fetch offchainworker test you made. I try to call api using the polkadotjs library, but an error occurs because the type is not defined. I have attached the code I tested. Likewise, the frontend you created will get the same error. Please define the type so that my javascript can work.

-----------------------------------my javascript code (block.js)-------------------------------------
var { ApiPromise, WsProvider, Keyring } = require('@polkadot/api');
var { cryptoWaitReady } = require('@polkadot/util-crypto');
var { stringToU8a, u8aToHex } = require('@polkadot/util');
//var { createType } = require('@polkadot/types');

const BOB = '1KvKReVmUiTc2LW2a4qyHsaJJ9eE9LRsywZkMk5hyBeyHgw';
const JAY = '15j4dg5GzsL1bw2U2AWgeyAk6QTxq43V7ZPbXdAmbVLjvDCK';

async function main () {

await cryptoWaitReady();
const provider = new WsProvider('ws://localhost::9944');
//const provider = new WsProvider('wss://rpc.polkadot.io');

const api = await ApiPromise.create({ provider});

console.log(api.genesisHash.toHex());

return

const [chain, nodeName, nodeVersion] = await Promise.all([
api.rpc.system.chain(),
api.rpc.system.name(),
api.rpc.system.version()
]);
console.log(You are connected to chain ${chain} using ${nodeName} v${nodeVersion});

var keyring = new Keyring({ type : 'sr25519'});
var alice = keyring.addFromUri('//Alice');

var transfer = api.tx.balances.transfer(JAY, 100);
//console.log("transfer:", transfer);

const hash = await transfer.signAndSend(BOB);
console.log('Transfer sent with hash', hash.toHex());

var PHRASE = 'entire material egg meadow latin bargain dutch coral blood melt acoustic thought';
var newPair = keyring.addFromUri(PHRASE);
var alice = keyring.addFromUri('//Alice', { name: 'Alice default' });
//console.log(${alice.meta.name}: has address ${alice.address} with publicKey [${alice.publicKey}]);

const message = stringToU8a('this is our message');
const signature = alice.sign(message);
const isValid = alice.verify(message, signature);
//console.log(The signature ${u8aToHex(signature)}, is ${isValid ? '' : 'in'}valid);

var chain1 = await api.rpc.system.chain();
var lastHeader = await api.rpc.chain.getHeader();
console.log(${chain1}: last block #${lastHeader.number} has hash ${lastHeader.hash});

console.log(api.consts.babe.epochDuration.toNumber());

let { data: { free: previousFree }, nonce: previousNonce } = await api.query.system.account(BOB);
console.log(${BOB} has a balance of ${previousFree}, nonce ${previousNonce});

}

main().catch(console.error).finally(() => process.exit());

-----------------error---------------------------------
[root@localhost api_test]# node block.js
Unable to resolve type StrVecBytes, it will fail on construction
Unknown types found, no types for StrVecBytes
Unable to decode Vec on index 1 createType(ExtrinsicV4):: createType(Call):: Struct: failed on 'args':: Cannot construct unknown type StrVecBytes
2020-09-26 06:21:12 RPC-CORE: getBlock(hash?: BlockHash): SignedBlock:: createType(SignedBlock):: Struct: failed on 'block':: Struct: failed on 'extrinsics':: createType(ExtrinsicV4):: createType(Call):: Struct: failed on 'args':: Cannot construct unknown type StrVecBytes
2020-09-26 06:21:12 API/INIT: Error: FATAL: Unable to initialize the API: createType(SignedBlock):: Struct: failed on 'block':: Struct: failed on 'extrinsics':: createType(ExtrinsicV4):: createType(Call):: Struct: failed on 'args':: Cannot construct unknown type StrVecBytes
at EventEmitter.value (/home/api_test/node_modules/@polkadot/api/base/Init.js:85:25)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:4282) UnhandledPromiseRejectionWarning: Error: FATAL: Unable to initialize the API: createType(SignedBlock):: Struct: failed on 'block':: Struct: failed on 'extrinsics':: createType(ExtrinsicV4):: createType(Call):: Struct: failed on 'args':: Cannot construct unknown type StrVecBytes
at EventEmitter.value (/home/api_test/node_modules/@polkadot/api/base/Init.js:85:25)
at process._tickCallback (internal/process/next_tick.js:68:7)
(node:4282) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:4282) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

The links in the Readme.md lead to 404s (at least for me).
e.g https://github.com/jimmychu0807/substrate-offchain-pricefetch/blob/master/docs/node/runtime/src/price_fetch.rs

shubhzdev@DESKTOP-VRPA3D6:~/substrate-offchain-worker-demo-dependabot-cargo-bumpalo-3.12.0$ ./scripts/docker_run.sh
*** Start Substrate node template ***
[+] Running 1/1
✔ Network substrate-offchain-worker-demo-dependabot-cargo-bumpalo-3120_default Removed 0.2s
[+] Building 0.0s (0/0)
[+] Creating 1/0
✔ Network substrate-offchain-worker-demo-dependabot-cargo-bumpalo-3120_default Created 0.0s
[+] Building 0.0s (0/0)
error: failed to download bumpalo v3.12.0

Caused by:
unable to get packages from source

Caused by:
failed to parse manifest at /home/shubhzdev/substrate-offchain-worker-demo-dependabot-cargo-bumpalo-3.12.0/node/./.cargo/registry/src/github.com-1ecc6299db9ec823/bumpalo-3.12.0/Cargo.toml

Caused by:
failed to parse the edition key

Caused by:
this version of Cargo is older than the 2021 edition, and only supports 2015 and 2018 editions.