This is a follow up for #49
Account.encrypt() and Account.decrypt() doctests got disabled because of getpass(). The idea is to write a doctest hardcoding the password to demonstrate both features in the doctest, refs @carver comments in #49 (comment) and #49 (review)
Apart from these 2 doctests 7 others got skipped (doctest: +SKIP) because they broke for different reason. We could also try investigating/fixing them in this task.

What is wrong

Hardware wallets are great!

How can they be implemented.

https://gist.github.com/bargst/5f896e4a5984593d43f1b4eb66f79d68

This appears to be a great starting point.

The base Account object should probably assume the default path. We should probably also expose an API for accessing sub-accounts at a different path assuming the ledger supports arbitrary BIP32 paths.

What was wrong?

Want to catch/prevent more issues like ethereum/web3.py#466 earlier.

How can it be fixed?

More hypothesis tests. Piper's suggestions (on ethereum/web3.py#477):

• A signed transaction's signature recovers to the expected sender address
• The raw transaction can be rlp decoded into a transaction object
• An rlp decoded transaction object's v,r,s match the v,r,s from the signature.
• The signature bytes are 65 bytes in length

The downside is that they can be pretty time-intensive. Just a test for the length of signature bytes added 7 seconds to a 53-second test suite.

Example:

--- a/tests/core/eth-module/test_accounts.py
+++ b/tests/core/eth-module/test_accounts.py
@@ -2,10 +2,19 @@
 
 import pytest
 
+from eth_keys.constants import (
+    SECPK1_N,
+)
 from eth_utils import (
     is_checksum_address,
 )
 
+from hypothesis import (
+    example,
+    given,
+    strategies as st,
+)
+
 from web3 import (
     Account,
     Web3,
@@ -268,6 +277,16 @@ def test_eth_account_sign(acct, message, key, expected_bytes, expected_hash, v,
     assert account.sign(message_text=message) == signed
 
 
+@given(st.binary(), st.integers(
+    min_value=Web3.toInt(hexstr='0x1' + '00' * 31),
+    max_value=SECPK1_N - 1
+    )
+)
+def test_message_signature_length(acct, message, key):
+    signed = acct.sign(message, key)
+    assert len(signed.signature) == 65
+
+
 @pytest.mark.parametrize(
     'txn, private_key, expected_raw_tx, tx_hash, r, s, v',
     (

If this is a bug report, please fill in the following sections.
If this is a feature request, delete and describe what you would like with examples.

What was wrong?

I created a Web3 instance which connects to an infura rinkeby node. Building a transaction works unless I call web3.eth.generateGasPrice() to determine an appropriate gas price.
The same scenario, but instead of infura I used a local geth POA node (own network) and instead of websockets I used HTTPProvider(...), worked without problems

Code that produced the error

        # ...
        self.networkid = 4
        self.admin_address = "0x..."
        wsprovider = "wss://" + network.lower() + ".infura.io/ws/v3/"\
                         + infura_projectid
        self._w3 = Web3(Web3.WebsocketProvider(wsprovider))
        self._w3.eth.setGasPriceStrategy(fast_gas_price_strategy)
        # inject the poa compatibility middleware to the innermost layer
        self._w3.middleware_stack.inject(geth_poa_middleware, layer=0)
        # Cache certain requests for avg_block_time (def. 240) seconds
        self._w3.middleware_stack.add(latest_block_based_cache_middleware)
        with open(contract_json) as confile:
            concontent = jload(confile)
        contract = self._w3.eth.\
                              contract(address=concontent["deployedAddress"],
                                          abi=concontent["abiDefinition"])
        nonce = self._w3.eth.getTransactionCount(self.admin_address)
        tx = contract.functions.createNew(lname, pts)\
                     .buildTransaction({
                        "from": self.admin_address,
                        "nonce": nonce,
                        "chainId": self.networkid,
                        "gasPrice": self._w3.eth.generateGasPrice()
                     })

Full error output

  File "/home/sea212/gproject/swagger_server/controllers/admins_controller.py", line 78, in create_lottery
    if not web3adapter.create(data):
  File "/home/sea212/gproject/swagger_server/web3adapter/web3adapter.py", line 154, in create
    "gasPrice": self._w3.eth.generateGasPrice()
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/contract.py", line 1228, in buildTransaction
    **self.kwargs
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/contract.py", line 1512, in build_transaction_for_function
    prepared_transaction = fill_transaction_defaults(web3, prepared_transaction)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/transactions.py", line 55, in fill_transaction_defaults
    default_val = default_getter(web3, transaction)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/transactions.py", line 26, in <lambda>
    'gas': lambda web3, tx: web3.eth.estimateGas(tx),
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/eth.py", line 304, in estimateGas
    [transaction],
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/manager.py", line 109, in request_blocking
    response = self._make_request(method, params)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/manager.py", line 92, in _make_request
    return request_func(method, params)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/cache.py", line 385, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/middleware/formatting.py", line 47, in apply_formatters
    formatted_params = formatter(params)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/eth_utils/functional.py", line 46, in inner
    return callback(fn(*args, **kwargs))
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/formatters.py", line 41, in apply_formatter_at_index
    yield formatter(item)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/rpc_abi.py", line 69, in apply_abi_formatters_to_dict
    [data[field] for field in fields],
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/abi.py", line 442, in map_abi_data
    return pipe(data, *pipeline)
  File "cytoolz/functoolz.pyx", line 589, in cytoolz.functoolz.pipe
  File "cytoolz/functoolz.pyx", line 565, in cytoolz.functoolz.c_pipe
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/abi.py", line 474, in data_tree_map
    return recursive_map(map_to_typed_data, data_tree)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/decorators.py", line 35, in wrapped
    wrapped_val = to_wrap(*args)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/formatters.py", line 115, in recursive_map
    items_mapped = map_collection(recurse, data)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/formatters.py", line 102, in map_collection
    return datatype(map(func, collection))
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/formatters.py", line 114, in recurse
    return recursive_map(func, item)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/decorators.py", line 35, in wrapped
    wrapped_val = to_wrap(*args)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/formatters.py", line 116, in recursive_map
    return func(items_mapped)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/abi.py", line 471, in map_to_typed_data
    return ABITypedData(func(*elements))
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/normalizers.py", line 50, in wrapper
    modified = to_wrap(abi_type, data)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/normalizers.py", line 104, in abi_int_to_hex
    return abi_type, hexstr_if_str(to_hex, data)
  File "cytoolz/functoolz.pyx", line 236, in cytoolz.functoolz.curry.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/web3/utils/encoding.py", line 213, in hexstr_if_str
    return to_type(primitive, hexstr=hexstr)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/eth_utils/decorators.py", line 75, in wrapper
    return to_wrap(*args, **kwargs)
  File "/home/sea212/.pyenv/versions/swagtest19/lib/python3.7/site-packages/eth_utils/conversions.py", line 41, in to_hex
    "or int.".format(repr(type(primitive)))
TypeError: Unsupported type: '<class 'float'>'.  Must be one of: bool, str, bytes, bytearrayor int.

Expected Result

This section may be deleted if the expectation is "don't crash".

Estimated gas price (integer)

Environment

# run this:
$ python -m eth_utils

# then copy the output here:
Python version:
3.7.1 (default, Nov 30 2018, 17:15:57) 
[GCC 7.3.0]

Operating System: Linux-4.15.0-43-generic-x86_64-with-debian-buster-sid

pip freeze result:
attrdict==2.0.0
backcall==0.1.0
bleach==3.1.0
certifi==2018.11.29
chardet==3.0.4
Click==7.0
clickclick==1.2.2
connexion==1.1.15
cytoolz==0.9.0.1
decorator==4.3.0
defusedxml==0.5.0
entrypoints==0.3
eth-abi==1.3.0
eth-account==0.3.0
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.1
eth-rlp==0.1.2
eth-typing==2.0.0
eth-utils==1.4.1
Flask==1.0.2
hexbytes==0.1.0
idna==2.8
inflection==0.3.1
ipykernel==5.1.0
ipython==7.2.0
ipython-genutils==0.2.0
ipywidgets==7.4.2
itsdangerous==1.1.0
jedi==0.13.2
Jinja2==2.10
jsonschema==2.6.0
jupyter==1.0.0
jupyter-client==5.2.4
jupyter-console==6.0.0
jupyter-core==4.4.0
lru-dict==1.1.6
MarkupSafe==1.1.0
mistune==0.8.4
nbconvert==5.4.0
nbformat==4.4.0
notebook==5.7.4
pandocfilters==1.4.2
parsimonious==0.8.1
parso==0.3.1
pexpect==4.6.0
pickleshare==0.7.5
prometheus-client==0.5.0
prompt-toolkit==2.0.7
ptyprocess==0.6.0
pycryptodome==3.7.2
Pygments==2.3.1
pymongo==3.7.2
python-dateutil==2.6.0
PyYAML==3.13
pyzmq==17.1.2
qtconsole==4.4.3
requests==2.21.0
rlp==1.1.0
Send2Trash==1.5.0
six==1.12.0
swagger-server==1.0.0
swagger-spec-validator==2.4.1
terminado==0.8.1
testpath==0.4.2
toolz==0.9.0
tornado==5.1.1
traitlets==4.3.2
urllib3==1.24.1
wcwidth==0.1.7
web3==4.8.2
webencodings==0.5.1
websockets==6.0
Werkzeug==0.14.1
widgetsnbextension==3.4.2

How can it be fixed?

Not regarded yet

Need to do necessary changes to support rlp>=1.0.0

Should target supporting full rlp>=0.1.0,<2 if it's not too difficult.

With the incoming work on #25 (in PR #26) -- it's time to have an abstract class defining the interface for a LocalAccount, plus some docs about which methods must be implemented. Specifically:

• address property is required
• signHash() is required
• signTransaction() is required
• __hash__() and __eq__() are required - so care should be taken that the account cannot be mutated to use a different private key.
• privateKey property is optional, since some sources refuse to reveal it (like a hardware wallet)
• encrypt() is optional, since some accounts cannot reveal the private key

(Also, I just noticed that the docs for encrypt in LocalAccount are wrong. See the definition in Account instead

The behavior is different in js and python. Figure out which one is right, and fix the other one.

First guess: one of the libraries calculates message length in unicode character points instead of number of bytes. Since the message length is embedded in the prefix, it would change the message and its hash.

See https://gist.github.com/holiman/6f57cb5b082b6712b9e8c19fe48ca960#gistcomment-2332077

Migrated from: ethereum/web3.py#583

What was wrong?

eth-keyfile supports passing in a kdf parameter to specify the key derivation function, but this api is not exposed via the web3.eth.account.encrypt API.

How can it be fixed?

Add an optional keyword parameter kdf to allow specifying what KDF you want.

Extra options:

Allow setting the default KDF using both environment variables and via a property on the web3.eth.account module.

web3.eth.account.defaultKDF = 'scrypt'

Not sure, this is more of a question probably:

Apparently, support for Python 3.5 was dropped with v0.4.0 via this commit.

This essentially breaks any apps running on PyPy since the latest PyPy release is still at Python 3.5.

The consequence of above for us is: we'll have to pin to eth-account==0.3.0 which is not ideal of course.

I am wondering: was there a hard reason to drop 3.5 support? Just asking ..

After I updated to the last version of eth-account I see this warning on my tests. At first I thought it would be just renaming the function, but I realized I cannot sign a custom hash anymore. I understand this can be a way of preventing users to sign hashes of transactions by mistake, but I think a developer should have the possibility of signing hashes and in my case I need it for some contracts that rely on signing hashes and not messages.

How can it be fixed?

I would propose not deprecating the old method or adding a raw SignableMessage that allows to sign hashes.

It could be possible that there's a way to do it and I didn't figure it out, sorry if that's the case

What was wrong?

Code that produced the error

When running the docs locally

$ make docs

I see the full documentation for methods and whatnot, but on readthedocs I don't see any of that?
https://eth-account.readthedocs.io/en/latest/eth_account.html

Expected Result

Method and class doc shows up at the link above.

Environment

Python version:
3.7.0 (default, Oct  2 2018, 09:18:58) 
[Clang 10.0.0 (clang-1000.11.45.2)]

Operating System: Darwin-18.0.0-x86_64-i386-64bit

pip freeze result:
alabaster==0.7.12
apipkg==1.5
appnope==0.1.0
argh==0.26.2
attrdict==2.0.0
attrs==18.2.0
Babel==2.6.0
backcall==0.1.0
bumpversion==0.5.3
certifi==2018.8.24
chardet==3.0.4
colorama==0.4.0
cytoolz==0.9.0.1
decorator==4.3.0
docopt==0.6.2
docutils==0.14
-e [email protected]:stefanmendoza/eth-account.git@adcd56cdeb4c65bd55c469c45ce78e2df1263a4d#egg=eth_account
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.2.0b3
eth-rlp==0.1.2
eth-typing==1.3.0
eth-utils==1.2.2
execnet==1.5.0
flake8==3.4.1
hexbytes==0.1.0
idna==2.7
imagesize==1.1.0
ipython==7.0.1
ipython-genutils==0.2.0
isort==4.3.4
jedi==0.13.1
Jinja2==2.10
MarkupSafe==1.0
mccabe==0.6.1
packaging==18.0
parso==0.3.1
pathtools==0.1.2
pexpect==4.6.0
pickleshare==0.7.5
pluggy==0.6.0
prompt-toolkit==2.0.6
ptyprocess==0.6.0
py==1.7.0
pycodestyle==2.3.1
pycryptodome==3.6.6
pyflakes==1.5.0
Pygments==2.2.0
pyparsing==2.2.2
pytest==3.3.2
pytest-forked==0.2
pytest-watch==4.2.0
pytest-xdist==1.23.2
pytz==2018.5
PyYAML==3.13
requests==2.19.1
rlp==1.0.3
simplegeneric==0.8.1
six==1.11.0
snowballstemmer==1.2.1
Sphinx==1.8.1
sphinx-rtd-theme==0.4.2
sphinxcontrib-websupport==1.1.0
toolz==0.9.0
tox==2.9.1
traitlets==4.3.2
urllib3==1.23
virtualenv==16.0.0
watchdog==0.9.0
wcwidth==0.1.7

How can it be fixed?

N/A

What was wrong?

Code that produced the error

from eth_account import Account
key = b'a' * 32
acct1 = Account.privateKeyToAccount(key)
acct2 = Account.privateKeyToAccount(key)
assert acct1 == acct2, "local accounts with the same private key should evaluate as equal"

Full error output

AssertionError: local accounts with the same private key should evaluate as equal

How can it be fixed?

Add an __eq__ to LocalAccount, test that both objects are exactly the same type, and that they have the same key.

Bonus: Make the private key attribute of LocalAccount immutable, and implement __hash__ as hash(<private_key>).

Faced a problem while following the instructions on https://buildmedia.readthedocs.org/media/pdf/eth-account/latest/eth-account.pdf

from eth_account import Account
acct = Account.create('KEYSMASH FJAFJKLDSKF7JKFDJ 1530')
acct.address  
acct.key
Traceback (most recent call last):

  File "<ipython-input-12-352e77122c2d>", line 4, in <module>
    acct.key

**AttributeError: 'LocalAccount' object has no attribute 'key'**

Although the account gets created , I am unable to retrieve the key as shown in the introductory documents ..

Could you please let me know how I can retrieve the private key for the account which got created?
Or am i doing something wrong?

Hello, I see, you've included support for EIP-1559, however when I call Account.sign_transaction method I fell into the chain of functions:
_utils.signing.sign_transaction_dict
-->_utils.legacy_transactions.serializable_unsigned_transaction_from_dict
--> assert_valid_fields.

And it all ends with an error:

TypeError: Transaction must include these fields: {'gasPrice'}.

How can I sign the new eip1559 transaction then? Do I need some other methods for eip1559 transactions?

EDIT:
Forgot to say, that I'm using the latest stable version: 0.5.5

STUB for a new account type to support trezor hardware wallets. Here's a starting point from the shadowlands app: https://github.com/kayagoban/shadowlands-core/blob/a82c2b32ed46464361d9be10ae9bd6a93039b462/shadowlands/credstick/trezor_ethdriver.py

• Tools like metamask use mnemonic passphrases for key backups.
• HD wallets are a widely used thing.

Supporting these types of accounts/wallets would likely be a useful feature.

Note that I'm not sure whether metamask uses bip39 style mnemonics so we should look into that.

Use case: local developer keys (for debugging on public networks with low-fund accounts)

Create an account where user provides an encrypted keystore, and Account.decrypt is used whenever the publicKey is required, providing a password to unlock the keystore for a 1-time sign operation. Optional functionality would be to cache this password for a given time period (similar to the personal_unlockAccount API), for which the user can accept or decline using [y/N].

For better user experience feedback, perhaps debug print the message or transaction to be signed before asking for the password or to use the cached password.

This may be more broadly useful setting dev experience expectations as a baseclass for hardware wallet interactions (#31), as the interactive prompt could delegate to the hardware signer instead.

What was wrong?

https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

BIP32 specifies Hierarchical Deterministic Wallets. These are common ways to generate private keys.

How can it be fixed?

Implement a new account class HDAccount which exposes the same API (potentially via subclassing) as LocalAccount, with additional APIs for generating child accounts.

Test cases should include any public test vectors that are available as part of the spec.

What was wrong?

Message signing should be very pluggable according to EIP191. Message encoding is all built-in to the (now confusingly named) defunct_hash_message().

How can it be fixed?

New message signing API that captures this concept:

class SignableMessage(NamedTuple):  # as defined by EIP-191
  version: HexBytes  # must be length 1
  header: HexBytes  # aka "version specific data"
  body: HexBytes  # aka "data to sign"

signable_message = encode_structured_data_message(json_str)
account.sign_message(signable_message)

The encoding method could have arbitrary inputs without changing the message signing API, like:

signable_message = encode_intended_validator(message, validator_address)
account.sign_message(signable_message)

It abstracts away the EIP-191 encoding process and message hashing inside sign_message(). A new recover_message() could work similarly.

The solution should make it trivial to experiment with or produce your own (sub)version of EIP-191, without any change to the eth-account library. (This is already possible by signing and recovering hashes of your own making, but it's not as accessible).

What was wrong?

Version 0.4.0 as released on PyPI has a dependency on eth-keys<0.3.0. However, eth-keys is now available in version 0.3.1, and some other Ethereum-related packages depend on >=0.3.0.

Expected Result

eth-keys should depend on eth-keys<0.4.0, as is currently the case in the master branch.

How can it be fixed?

Probably by bumping the eth-account version to 0.4.1 and updating the dependency

What was wrong?

Code that produced the error

I'm trying to replay mainnet transactions locally.

I've adapted this code from eth-accounts to recover raw transactions from mainnet transaction data. I'm using 2 functions from eth-accounts: serializable_unsigned_transaction_from_dict and encode_transaction.

import pytest
import web3
import eth_account
import rlp
from eth.vm.forks.muir_glacier.transactions import MuirGlacierTransaction

def recover_raw_transaction(tx) -> str:
    """Recover raw transaction for replay.

    Adapted from: https://github.com/ethereum/eth-account/blob/1d26f44f6075d6f283aeaeff879f4508c9a228dc/eth_account/_utils/signing.py#L28-L42
    """
    transaction = {
        "to": tx["to"],
        "gas": tx["gas"],
        "gasPrice": tx["gasPrice"],
        "value": tx["value"],
        "nonce": tx["nonce"],
    }
    if "data" in tx:
        transaction["data"] = tx["data"]

    v = tx["v"]
    r = int.from_bytes(tx["r"], "big")
    s = int.from_bytes(tx["s"], "big")
    unsigned_transaction = serializable_unsigned_transaction_from_dict(transaction)
    return encode_transaction(unsigned_transaction, vrs=(v, r, s))

However, as I'm trying to use this on a specific mainnet transaction, I'm getting a different raw_tx than in Etherscan.

def test_extract_from_account():
    w3 = web3.Web3(web3.HTTPProvider("<your favorite mainnet url>"))
    tx = w3.eth.getTransaction("0x2133167f62f03f65e07210640530373aad2e743ee9e2c025b9abec4e8fe2133d")
    raw_tx = recover_raw_transaction(tx)
    print(raw_tx)

Full error output

This is the raw transaction that I'm getting:

>>> a = b'\xf8o\x82\x0e\xcc\x85tjR\x88\x00\x83\x02\xe9\xae\x94z%\rV0\xb4\xcfS\x979\xdf,]\xac\xb4\xc6Y\xf2H\x8d\x88\x05\x8d\x15\xe1v(\x00\x00\x80&\xa0\xa3\xeb\xf3m68\xc6\xd1\xfc\xd8@\xaf\x06\xa9C\x94\r\xa8h\xcf\xde\x07\xa9\xe3\xea\xfa\x03\xd46\xeav\x8a\xa0f,\xe8\xcf\xe2\x11\x0cj`\xcd\xc97 \x19\xb46\xb3}\x8f\xa2\xfd\x8f9\xaf\xdb\xa3\n\x9dd\x1e\x80q'
>>> a.hex()
'f86f820ecc85746a5288008302e9ae947a250d5630b4cf539739df2c5dacb4c659f2488d88058d15e1762800008026a0a3ebf36d3638c6d1fcd840af06a943940da868cfde07a9e3eafa03d436ea768aa0662ce8cfe2110c6a60cdc9372019b436b37d8fa2fd8f39afdba30a9d641e8071' 

Expected Result

See [Etherscan].

Returned Raw Transaction Hex :

0xf90154820ecc85746a5288008302e9ae947a250d5630b4cf539739df2c5dacb4c659f2488d88058d15e176280000b8e47ff36ab50000000000000000000000000000000000000000000000e4230d4f00b9fccc9a000000000000000000000000000000000000000000000000000000000000008000000000000000000000000014b95ed55c0825a30c5bf6d4905379e06749b117000000000000000000000000000000000000000000000000000000005f5783460000000000000000000000000000000000000000000000000000000000000002000000000000000000000000c02aaa39b223fe8d0a0e5c4f27ead9083c756cc2000000000000000000000000ff20817765cb7f73d4bde2e66e067e58d11095c226a0a3ebf36d3638c6d1fcd840af06a943940da868cfde07a9e3eafa03d436ea768aa0662ce8cfe2110c6a60cdc9372019b436b37d8fa2fd8f39afdba30a9d641e8071 

Environment

aniso8601==7.0.0
appdirs==1.4.4
asttokens==2.0.3
attrdict==2.0.1
attrs==20.2.0
base58==2.0.1
bitarray==1.2.2
black==20.8b1
blake2b-py==0.1.3
cached-property==1.5.1
certifi==2020.6.20
chardet==3.0.4
click==7.1.2
cytoolz==0.10.1
eth-abi==2.1.1
eth-account==0.5.3
eth-bloom==1.0.3
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.0
eth-tester==0.5.0b2
eth-typing==2.2.2
eth-utils==1.9.5
fastdiff==0.2.0
Flask==1.1.2
Flask-Cors==3.0.9
Flask-GraphQL==2.0.1
future==0.18.2
gevent==20.6.2
graphene==2.1.8
graphql-core==2.3.2
graphql-relay==2.0.1
graphql-server-core==1.2.0
greenlet==0.4.16
gunicorn==20.0.4
hexbytes==0.2.1
idna==2.10
importlib-metadata==1.7.0
iniconfig==1.0.1
ipfshttpclient==0.6.1
isort==5.5.1
itsdangerous==1.1.0
Jinja2==2.11.2
jsonschema==3.2.0
lru-dict==1.1.6
MarkupSafe==1.1.1
more-itertools==8.5.0
multiaddr==0.0.9
mypy==0.782
mypy-extensions==0.4.3
netaddr==0.8.0
packaging==20.4
parsimonious==0.8.1
pathspec==0.8.0
pluggy==0.13.1
promise==2.3
protobuf==3.13.0
py==1.9.0
py-ecc==4.0.0
py-evm==0.3.0a19
py-geth==2.4.0
py-solc===3.2.0-fixedstdin
py-solc-x==1.0.0
pycryptodome==3.9.8
pyethash==0.1.27
pyevmasm==0.2.3
pyparsing==2.4.7
pyrsistent==0.16.0
pysha3==1.0.2
pytest==6.0.1
python-json-logger==0.1.11
regex==2020.7.14
requests==2.24.0
rlp==2.0.0a1
rusty-rlp==0.1.15
Rx==1.6.1
semantic-version==2.8.5
six==1.15.0
snapshottest==0.5.1
sortedcontainers==2.2.2
termcolor==1.1.0
toml==0.10.1
toolz==0.10.0
trie==2.0.0a4
typed-ast==1.4.1
typing-extensions==3.7.4.3
urllib3==1.25.10
varint==1.0.2
vyper==0.2.4
wasmer==0.4.1
wcwidth==0.1.8
web3==5.12.1
websockets==8.1
Werkzeug==1.0.1
zipp==3.1.0
zope.event==4.4
zope.interface==5.1.0

How can it be fixed?

• Version: 5.13.1
• Python: 3.8.6
• OS: win 10
• pip freeze output

attrs==20.3.0
base58==2.0.1
beautifulsoup4==4.9.3
bitarray==1.2.2
cachetools==4.1.1
certifi==2020.11.8
cffi==1.14.4
chardet==3.0.4
click==7.1.2
cytoolz==0.11.0
eth-abi==2.1.1
eth-account==0.5.4
eth-hash==0.2.0
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-typing==2.2.2
eth-utils==1.9.5
Flask==1.1.2
google-api-core==1.23.0
google-auth==1.23.0
google-cloud-core==1.4.4
google-cloud-logging==2.0.1
google-cloud-pubsub==2.1.0
google-cloud-storage==1.33.0
google-crc32c==1.0.0
google-resumable-media==1.1.0
googleapis-common-protos==1.52.0
grpc-google-iam-v1==0.12.3
grpcio==1.34.0
hexbytes==0.2.1
idna==2.10
ipfshttpclient==0.7.0a1
itsdangerous==1.1.0
Jinja2==2.11.2
jsonschema==3.2.0
libcst==0.3.15
lru-dict==1.1.6
MarkupSafe==1.1.1
multiaddr==0.0.9
mypy-extensions==0.4.3
netaddr==0.8.0
parsimonious==0.8.1
proto-plus==1.11.0
protobuf==3.14.0
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycryptodome==3.9.9
pypiwin32==223
pyrsistent==0.17.3
pytz==2020.4
pywin32==300
PyYAML==5.3.1
requests==2.24.0
rlp==2.0.1
rsa==4.6
signalr-client-threads==0.0.12
six==1.15.0
soupsieve==2.1
sseclient==0.0.27
toolz==0.11.1
typing-extensions==3.7.4.3
typing-inspect==0.6.0
urllib3==1.25.11
varint==1.0.2
web3==5.13.1
websocket-client==0.57.0
websockets==8.1
Werkzeug==1.0.1

What was wrong?

I've been trying to understand EIP-712 and was signing permits for Uniswap, which worked fine, however as I was testing it, I suddenly got an error:

web3.exceptions.ValidationError:
Could not identify the intended function with name `removeLiquidityETHWithPermit`, positional argument(s) of type `(<class 'str'>, <class 'int'>, <class 'int'>, <class 'int'>, <class 'str'>, <class 'int'>, <class 'bool'>, <class 'int'>, <class 'str'>, <class 'str'>)` and keyword argument(s) of type `{}`.
Found 1 function(s) with the name `removeLiquidityETHWithPermit`: ['removeLiquidityETHWithPermit(address,uint256,uint256,uint256,address,uint256,bool,uint8,bytes32,bytes32)']
Function invocation failed due to no matching argument types.

which was really weird, as the parameters seem to be correct and it was working fine multiple times just 5 sec before that... I've been trying to figure out what's wrong and I've realized that whenever I sign a message, and the message ends up with an 'r' value that has only 63 chars, e.g.

0x989d37dca7a52486cfa871a165e2263aa355ad99dfca670fe918f59d4e84be0

it causes an error instead of the function adding a 0 before first 9, thus making it

0x0989d37dca7a52486cfa871a165e2263aa355ad99dfca670fe918f59d4e84be0

and building a proper hex string out of it

just to make sure it would actually work (and the error did not occur during the message signing) I tried to "write contract" via etherscan and use the 63 char hex string (the one that caused an error), which did result in a successful transaction that did have an additional 0 after 0x (I assume etherscan added it before building the transaction)

https://ropsten.etherscan.io/tx/0xe657b59e3a946f90ba3226488878fd5be09ea6af250b896c3f323037c1f9888b

when I toyed with deadline, amount or nonce variables in the code, and the r value ended up with 64 chars, I did not get the error for the function and could build it properly, however the error re-emerged with any 63 char hex string...

from web3 import Web3;
from eth_account import messages;
import json;

w3 = Web3(Web3.HTTPProvider(""));

amount = 22593420516831854593;
deadline = 1609966542;
nonce = 11;

# address = ""; # sender address
# privkey = ""; # sender private key

# token = ""; # token used in WETH-TKN pair

uniswap = "0x7a250d5630B4cF539739dF2C5dAcb4c659F2488D";
factory = "0x5C69bEe701ef814a2B6a3EDD4B1652CB9cc5aA6f";
weth = "0xc778417E063141139Fce010982780140Aa0cD5Ab"; # ropsten WETH

with open("./abi/uniswap.abi", "r") as f: abi = json.load(f);
uniswap_contract = w3.eth.contract(address=uniswap, abi=json.dumps(abi));

with open("./abi/factory.abi", "r") as f: abi = json.load(f);
factory_contract = w3.eth.contract(address=factory, abi=json.dumps(abi));

with open("./abi/uniswap_pair.abi", "r") as f: pair_abi = json.load(f);

exchange_address = factory_contract.functions.getPair(weth, token).call();
pair_contract = w3.eth.contract(address=exchange_address, abi=json.dumps(pair_abi));

domain = [
	{ "name": "name", "type": "string" },
	{ "name": "version", "type": "string" },
	{ "name": "chainId", "type": "uint256" },
	{ "name": "verifyingContract", "type": "address" }
];

permit = [
	{ "name": "owner", "type": "address" },
	{ "name": "spender", "type": "address" },
	{ "name": "value", "type": "uint256" },
	{ "name": "nonce", "type": "uint256" },
	{ "name": "deadline", "type": "uint256" }
];

domain_data = dict(
	name = "Uniswap V2",
	version = "1",
	chainId = 3,
	verifyingContract = exchange_address
);

permit_data = dict(
	owner = address,
	spender = uniswap,
	value = amount,
	nonce = nonce,
	deadline = deadline
);

data = dict(
	types = dict(
		EIP712Domain = domain,
		Permit = permit
	),
	domain = domain_data,
	primaryType = "Permit",
	message = permit_data
);

encoded_data = messages.encode_structured_data(data);
signed_tx = w3.eth.account.sign_message(encoded_data, private_key=privkey);

print(signed_tx.r);
print(signed_tx.s);
print(Web3.toHex(signed_tx.r));
print(Web3.toHex(signed_tx.s));

f = uniswap_contract.functions.removeLiquidityETHWithPermit(token, amount, 0, 0, address, deadline, False, signed_tx.v, Web3.toHex(signed_tx.r), Web3.toHex(signed_tx.s));

tx = f.buildTransaction({"from":address, "gas":300000, "gasPrice":10000000000, "nonce":w3.eth.getTransactionCount(address, "pending")});
signed_tx = w3.eth.account.sign_transaction(tx, private_key=privkey);

res = w3.eth.sendRawTransaction(signed_tx.rawTransaction);
print(res.hex());

How can it be fixed?

My guess is that when the library expects a 'bytes32' argument type, it does expect a hex string that has exactly 64(+2) chars, however it should add leading 0s after 0x if the length is less than 64 and an actual hex string was provided (0x...)

What was wrong?

An unsigned transaction's hash is wrong and not signable

transaction = DynamicFeeTransaction.from_dict(txn)
encoded_txn = transaction.hash()   #  <---not right hash

My transaction does not have V, R, or S (it is unsigned).
It also does not have any access lists.
The tests and docs are not helping either:

• Every test case for test_typed_transaction::test_hash uses already signed transactions
• Every test case (same as above) uses access lists

The hash I am getting does not seem correct because of its length and because the Ledger dongle does not accept it

MORE

This is my transaction dict:

{'chainId': 61, 'to': b'', 'value': 0, 'data': b'`\x80`@R4\x80\x15a\x00\x10W`\x00\x80\xfd[P3`\x00\x80a\x01\x00\n\x81T\x81s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x02\x19\x16\x90\x83s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x02\x17\x90UP`\x01`\x03`\x00a\x01\x00\n\x81T\x81`\xff\x02\x19\x16\x90\x83\x15\x15\x02\x17\x90UPa\x07\xca\x80a\x00{`\x009`\x00\xf3\xfe`\x80`@R`\x046\x10a\x00pW`\x005`\xe0\x1c\x80c>G\xd6\xf3\x11a\x00NW\x80c>G\xd6\xf3\x14a\x00\xe9W\x80c\x8d\xa5\xcb[\x14a\x01NW\x80c\xb6\rB\x88\x14a\x01\x8fW\x80c\xdc\r=\xff\x14a\x01\x99Wa\x00pV[\x80c\x12)\xdc\x9e\x14a\x00uW\x80c#\x8d\xaf\xe0\x14a\x00\xb2W\x80c<\xcf\xd6\x0b\x14a\x00\xdfW[`\x00\x80\xfd[4\x80\x15a\x00\x81W`\x00\x80\xfd[Pa\x00\xb0`\x04\x806\x03` \x81\x10\x15a\x00\x98W`\x00\x80\xfd[\x81\x01\x90\x80\x805\x15\x15\x90` \x01\x90\x92\x91\x90PPPa\x01\xfeV[\x00[4\x80\x15a\x00\xbeW`\x00\x80\xfd[Pa\x00\xc7a\x02\xdcV[`@Q\x80\x82\x15\x15\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[a\x00\xe7a\x02\xefV[\x00[4\x80\x15a\x00\xf5W`\x00\x80\xfd[Pa\x018`\x04\x806\x03` \x81\x10\x15a\x01\x0cW`\x00\x80\xfd[\x81\x01\x90\x80\x805s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90` \x01\x90\x92\x91\x90PPPa\x05\x10V[`@Q\x80\x82\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[4\x80\x15a\x01ZW`\x00\x80\xfd[Pa\x01ca\x05(V[`@Q\x80\x82s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[a\x01\x97a\x05LV[\x00[4\x80\x15a\x01\xa5W`\x00\x80\xfd[Pa\x01\xd2`\x04\x806\x03` \x81\x10\x15a\x01\xbcW`\x00\x80\xfd[\x81\x01\x90\x80\x805\x90` \x01\x90\x92\x91\x90PPPa\x06pV[`@Q\x80\x82s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81R` \x01\x91PP`@Q\x80\x91\x03\x90\xf3[`\x00\x80T\x90a\x01\x00\n\x90\x04s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x163s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x14a\x02\xbfW`@Q\x7f\x08\xc3y\xa0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x81R`\x04\x01\x80\x80` \x01\x82\x81\x03\x82R`\x0b\x81R` \x01\x80\x7f!authorized\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x81RP` \x01\x91PP`@Q\x80\x91\x03\x90\xfd[\x80`\x03`\x00a\x01\x00\n\x81T\x81`\xff\x02\x19\x16\x90\x83\x15\x15\x02\x17\x90UPPV[`\x03`\x00\x90T\x90a\x01\x00\n\x90\x04`\xff\x16\x81V[`\x00\x80T\x90a\x01\x00\n\x90\x04s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x163s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x14a\x03\xb0W`@Q\x7f\x08\xc3y\xa0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x81R`\x04\x01\x80\x80` \x01\x82\x81\x03\x82R`\x0b\x81R` \x01\x80\x7f!authorized\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x81RP` \x01\x91PP`@Q\x80\x91\x03\x90\xfd[`\x03`\x00\x90T\x90a\x01\x00\n\x90\x04`\xff\x16a\x03\xc9W`\x00\x80\xfd[3s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16a\x08\xfcG\x90\x81\x15\x02\x90`@Q`\x00`@Q\x80\x83\x03\x81\x85\x88\x88\xf1\x93PPPP\x15\x80\x15a\x04\x0fW=`\x00\x80>=`\x00\xfd[P`\x00[`\x02\x80T\x90P\x81\x10\x15a\x04\xafW`\x00`\x02\x82\x81T\x81\x10a\x04/W\xfe[\x90`\x00R` `\x00 \x01`\x00\x90T\x90a\x01\x00\n\x90\x04s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x90P`\x00`\x01`\x00\x83s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81R` \x01\x90\x81R` \x01`\x00 \x81\x90UPP\x80\x80`\x01\x01\x91PPa\x04\x13V[P`\x00g\xff\xff\xff\xff\xff\xff\xff\xff\x81\x11\x80\x15a\x04\xc8W`\x00\x80\xfd[P`@Q\x90\x80\x82R\x80` \x02` \x01\x82\x01`@R\x80\x15a\x04\xf7W\x81` \x01` \x82\x02\x806\x837\x80\x82\x01\x91PP\x90P[P`\x02\x90\x80Q\x90` \x01\x90a\x05\r\x92\x91\x90a\x06\xacV[PV[`\x01` R\x80`\x00R`@`\x00 `\x00\x91P\x90PT\x81V[`\x00\x80T\x90a\x01\x00\n\x90\x04s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81V[`\x03`\x00\x90T\x90a\x01\x00\n\x90\x04`\xff\x16a\x05eW`\x00\x80\xfd[`\x004\x11a\x05\xbeW`@Q\x7f\x08\xc3y\xa0\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x81R`\x04\x01\x80\x80` \x01\x82\x81\x03\x82R`#\x81R` \x01\x80a\x07r`#\x919`@\x01\x91PP`@Q\x80\x91\x03\x90\xfd[4`\x01`\x003s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81R` \x01\x90\x81R` \x01`\x00 `\x00\x82\x82T\x01\x92PP\x81\x90UP`\x023\x90\x80`\x01\x81T\x01\x80\x82U\x80\x91PP`\x01\x90\x03\x90`\x00R` `\x00 \x01`\x00\x90\x91\x90\x91\x90\x91a\x01\x00\n\x81T\x81s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x02\x19\x16\x90\x83s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x02\x17\x90UPV[`\x02\x81\x81T\x81\x10a\x06}W\xfe[\x90`\x00R` `\x00 \x01`\x00\x91PT\x90a\x01\x00\n\x90\x04s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x81V[\x82\x80T\x82\x82U\x90`\x00R` `\x00 \x90\x81\x01\x92\x82\x15a\x07%W\x91` \x02\x82\x01[\x82\x81\x11\x15a\x07$W\x82Q\x82`\x00a\x01\x00\n\x81T\x81s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x02\x19\x16\x90\x83s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x16\x02\x17\x90UP\x91` \x01\x91\x90`\x01\x01\x90a\x06\xccV[[P\x90Pa\x072\x91\x90a\x076V[P\x90V[[\x80\x82\x11\x15a\x07mW`\x00\x81\x81a\x01\x00\n\x81T\x90s\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\xff\x02\x19\x16\x90UP`\x01\x01a\x077V[P\x90V\xfeFund amount must be greater than 0.\xa2dipfsX"\x12  \xbcL\xe7gG\xd6\x88*\xee\x91\x96\x8b\x7fV\xed\xdb\xe5 \xd4\x0eJ\xb6\x13s\x97w8\xe6w.\x92dsolcC\x00\x06\x0c\x003', 'accessList': [], 'nonce': 0, 'gas': 537114, 'maxPriorityFeePerGas': 875000000, 'maxFeePerGas': 0, 'type': 2}

This is the return value from hash():

b' U:\xe9\x00\xe3O\xfc\\>\x82\x92\x97\xaf9\x9d\x1a\xed\x0f)\xc6%\xbf\\\x08\xc4\x95\t\xce.\xcc7'

Environment

# run this:
$ python -m eth_utils

# then copy the output here:
Python version:
3.7.9 (v3.7.9:13c94747c7, Aug 15 2020, 01:31:08) 
[Clang 6.0 (clang-600.0.57)]

Operating System: Darwin-21.3.0-x86_64-i386-64bit

pip freeze result:
aiohttp==3.8.1
aiosignal==1.2.0
alabaster==0.7.12
-e git+ssh://[email protected]/ApeWorX/ape-ledger.git@a7d0fbc2ba083f75054e02fe9a9622d5c895acbc#egg=ape_ledger
ape-solidity==0.1.0b5
appnope==0.1.2
argcomplete==1.12.3
async-timeout==4.0.2
asynctest==0.13.0
attrs==21.4.0
Babel==2.9.1
backcall==0.2.0
backports.cached-property==1.0.1
base58==2.1.1
bitarray==1.2.2
black==21.12b0
bleach==4.1.0
cached-property==1.5.2
certifi==2021.10.8
cffi==1.15.0
cfgv==3.3.1
charset-normalizer==2.0.11
click==8.0.3
colorama==0.4.4
commitizen==2.19.0
commonmark==0.9.1
coverage==6.3.1
cytoolz==0.11.2
dataclassy==0.10.4
decli==0.5.2
decorator==5.1.1
Deprecated==1.2.13
distlib==0.3.4
docopt==0.6.2
docutils==0.16
ecdsa==0.17.0
eip712==0.1.0
eth-abi==2.1.1
eth-account==0.5.7
-e git+ssh://[email protected]/unparalleled-js/ape.git@93f5987b50ff5964f32f73d5dd5f4562d7ace691#egg=eth_ape
eth-bloom==1.0.4
eth-hash==0.3.2
eth-keyfile==0.5.1
eth-keys==0.3.4
eth-rlp==0.2.1
eth-tester==0.6.0b6
eth-typing==2.3.0
eth-utils==1.10.0
ethpm-types==0.1.0b7
execnet==1.9.0
fastecdsa==2.2.3
filelock==3.4.2
flake8==3.9.2
flake8-breakpoint==1.1.0
flake8-plugin-utils==1.3.2
flake8-print==4.0.0
frozenlist==1.3.0
hexbytes==0.2.2
hidapi==0.10.1
hypothesis==6.36.1
hypothesis-jsonschema==0.19.0
identify==2.4.8
idna==3.3
imagesize==1.3.0
importlib-metadata==4.10.1
iniconfig==1.1.1
ipdb==0.13.9
ipfshttpclient==0.8.0a2
ipython==7.31.1
isort==5.10.1
jedi==0.18.1
Jinja2==3.0.3
jsonschema==3.2.0
keyring==23.5.0
lru-dict==1.1.7
markdown-it-py==2.0.1
MarkupSafe==2.0.1
matplotlib-inline==0.1.3
mccabe==0.6.1
mdit-py-plugins==0.3.0
mdurl==0.1.0
mpmath==1.2.1
multiaddr==0.0.9
multidict==6.0.2
mypy==0.931
mypy-extensions==0.4.3
myst-parser==0.16.0
netaddr==0.8.0
nodeenv==1.6.0
numpy==1.21.5
packaging==20.9
pandas==1.3.5
pandas-stubs==1.2.0.49
parsimonious==0.8.1
parso==0.8.3
pathspec==0.9.0
pexpect==4.8.0
pickleshare==0.7.5
pkginfo==1.8.2
platformdirs==2.4.1
pluggy==0.13.1
pockets==0.9.1
pre-commit==2.17.0
prompt-toolkit==3.0.26
protobuf==3.19.4
ptyprocess==0.7.0
py==1.11.0
py-ecc==4.1.0
py-evm==0.5.0a3
py-geth==3.7.0
py-solc-x==1.1.1
pycodestyle==2.7.0
pycparser==2.21
pycryptodome==3.14.1
pydantic==1.9.0
pyethash==0.1.27
pyflakes==2.3.1
pygit2==1.8.0
PyGithub==1.55
Pygments==2.11.2
PyJWT==2.3.0
PyNaCl==1.5.0
pyparsing==3.0.7
pyrsistent==0.18.1
pysha3==1.0.2
pytest==6.2.5
pytest-cov==3.0.0
pytest-forked==1.4.0
pytest-mock==3.7.0
pytest-watch==4.2.0
pytest-xdist==2.5.0
python-dateutil==2.8.2
pytz==2021.3
PyYAML==5.4.1
questionary==1.10.0
readme-renderer==32.0
requests==2.27.1
requests-toolbelt==0.9.1
rfc3986==2.0.0
rich==10.16.2
rlp==2.0.1
semantic-version==2.8.5
setuptools-scm==6.4.2
singledispatchmethod==1.0
six==1.16.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
Sphinx==3.5.4
sphinx-click==3.1.0
sphinx-rtd-theme==0.5.2
sphinxcontrib-applehelp==1.0.2
sphinxcontrib-devhelp==1.0.2
sphinxcontrib-htmlhelp==2.0.0
sphinxcontrib-jsmath==1.0.1
sphinxcontrib-napoleon==0.7
sphinxcontrib-qthelp==1.0.3
sphinxcontrib-serializinghtml==1.1.5
sympy==1.9
termcolor==1.1.0
tokenlists==0.1.1
toml==0.10.2
tomli==1.2.3
tomlkit==0.9.0
toolz==0.11.2
tqdm==4.62.3
traitlets==5.1.1
trie==2.0.0a5
twine==3.8.0
typed-ast==1.5.2
types-PyYAML==6.0.4
types-requests==2.27.8
types-urllib3==1.26.9
typing-extensions==3.10.0.2
urllib3==1.26.8
varint==1.0.2
virtualenv==20.13.0
vvm==0.1.0
watchdog==2.1.6
wcwidth==0.2.5
web3==5.27.0
webencodings==0.5.1
websockets==9.1
wrapt==1.13.3
yarl==1.7.2
zipp==3.7.0

Add snake_case options for all public APIs. Raise warnings and add notes to all the other methods, to indicate they are deprecated. This minor (breaking) bump to v0.4 will introduce the deprecation. The next minor (breaking) bump to v0.5 will remove all camelCase methods.

What was wrong?

When importing eth_account, I get the error that the module Cryto is not found.
I am on OSX Big Sur and my python version is v3.8
I have installed these crypto packages:

crypto==1.4.1
pycrypto==2.6.1
pycryptodome==3.11.0

Seems to be an issue with Crypto library being imported and capitalized?

Code that produced the error

from eth_account.messages import defunct_hash_message 

Full error output

Traceback (most recent call last):
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/rest_framework/settings.py", line 177, in import_from_string
    return import_string(val)
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/django/utils/module_loading.py", line 17, in import_string
    module = import_module(module_path)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/Users/kcelica/Code/Django-for-web3/core/auth/backends.py", line 2, in <module>
    from eth_account.messages import defunct_hash_message
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/eth_account/__init__.py", line 1, in <module>
    from eth_account.account import (
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/eth_account/account.py", line 11, in <module>
    from eth_keyfile import (
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/eth_keyfile/__init__.py", line 7, in <module>
    from eth_keyfile.keyfile import (  # noqa: F401
  File "/Users/kcelica/Code/Django-for-web3/env/lib/python3.8/site-packages/eth_keyfile/keyfile.py", line 5, in <module>
    from Crypto import Random
ModuleNotFoundError: No module named 'Crypto'

Expected Result

I see others have fixed their issue by manually editing their file names in site-packages but this is not a long-term solution or makes it easy to use this code anywhere especially when deployed in a CI/CD setup.

Environment

# run this:
$ python -m eth_utils

# then copy the output here:
$  python -m eth_utils
Python version:
3.8.5 (v3.8.5:580fbb018f, Jul 20 2020, 12:11:27)
[Clang 6.0 (clang-600.0.57)]

Operating System: macOS-10.16-x86_64-i386-64bit

pip freeze result:
aiohttp==3.8.1
aiosignal==1.2.0
asgiref==3.4.1
async-timeout==4.0.1
attrs==21.2.0
base58==2.1.1
bitarray==1.2.2
certifi==2021.10.8
charset-normalizer==2.0.7
crypto==1.4.1
cytoolz==0.11.2
Django==3.2.8
django-cors-headers==3.10.0
djangorestframework==3.12.4
djangorestframework-simplejwt==5.0.0
eth-abi==2.1.1
eth-account==0.5.6
eth-hash==0.3.2
eth-keyfile==0.5.1
eth-keys==0.3.3
eth-rlp==0.2.1
eth-typing==2.2.2
eth-utils==1.10.0
frozenlist==1.2.0
hexbytes==0.2.2
idna==3.3
ipfshttpclient==0.8.0a2
jsonschema==3.2.0
lru-dict==1.1.7
multiaddr==0.0.9
multidict==5.2.0
Naked==0.1.31
netaddr==0.8.0
parsimonious==0.8.1
protobuf==3.19.1
pycrypto==2.6.1
pycryptodome==3.11.0
PyJWT==2.3.0
pyrsistent==0.18.0
pytz==2021.3
PyYAML==6.0
requests==2.26.0
rlp==2.0.1
shellescape==3.8.1
six==1.16.0
sqlparse==0.4.2
toolz==0.11.2
typing_extensions==4.0.0
urllib3==1.26.7
varint==1.0.2
web3==5.25.0
websockets==9.1
yarl==1.7.2

None

When I call w3.eth.account.signTransaction(tx, key)

I am getting the following error:

TypeError: from field must match key's 0xe6fAfd26257963055c93fc8a38Dd4554BB157349, but it was 0xe6fafd26257963055c93fc8a38dd4554bb157349

Obviously the key converts to the right address, but some of the letters are caps while some are not. What can I do about it?

It would be nice to implement support for the following EIPs:
EIP-2718 - Typed Transaction Envelopes
EIP-2930 - Optional access lists

Probably similar to: gakonst/ethers-rs#265

If this is a bug report, please fill in the following sections.
If this is a feature request, delete and describe what you would like with examples.

What was wrong?

Code that produced the error

CODE_TO_REPRODUCE 

Full error output

ERROR_HERE 

Expected Result

This section may be deleted if the expectation is "don't crash".

EXPECTED_RESULT

Environment

# run this:
$ python -m eth_utils

# then copy the output here:
OUTPUT_HERE

How can it be fixed?

Fill this section in if you know how this could or should be fixed.

What was wrong?

• bitarray dependency seems to be the cause of many users not being able to install web3.py:
• ethereum/web3.py#1761
• ethereum/web3.py#1788
• ethereum/web3.py#2133
• ethereum/web3.py#2147
• ethereum/web3.py#2170
• ethereum/web3.py#2171...

... and many cases on Discord

It seems to only be used in hdaccount support. We should investigate if we can remove this dependency while still supporting hdaccount logic.

How can it be fixed?

1. Can we somehow remove bitarray dependency and keep hdaccount support?
2. Does updating the bitarray version resolve these issues?

I'm not confident in option 2 as I believe some users also tried to manually pip install bitarray and also had issues but I have not looked too far into this so I'll leave this option up.

In addition to accepting hex strings and bytes, eth_account should accept private keys generated by eth_keys.

What was wrong?

I'm trying to sign some data using EIP-712 with their Web3 client (Using Metamask in my case), and then recover the address on the backend to verify that a user is actually the owner of that address. However, the address I get back from Account.recover_message doesn't match it.

Code that produced the error

Client JS

const chainId = await web3.eth.net.getId();
const [address] = await web3.eth.getAccounts();
const data = {
  types: {
    EIP712Domain: [
      { name: 'name', type: 'string' },
      { name: 'version', type: 'string' },
      { name: 'chainId', type: 'uint256' },
      { name: 'verifyingContract', type: 'string' },
    ],
    authorization: [
      { name: 'userid', type: 'uint256' },
      { name: 'point', type: 'string' },
    ],
  },
  domain: {
    name: 'test.test',
    version: '1',
    chainId,
    verifyingContract: '0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC',
  },
  primaryType: 'authorization',
  message: {
    userid: 123,
    point: 'abc',
  },
};

const signature = await web3.currentProvider.send('eth_signTypedData_v3', [
  address,
  JSON.stringify(data),
]);

// send signature and data to python backend...

Server Python

encoded = encode_structured_data(primitive=data)
addr = Account.recover_message(encoded, signature=signature)
# python addr !== js address

Let me know if I'm not doing this correctly, or if there are known differences between Metamask's EIP-712 format and this library's.

What was wrong?

Mnemonics only work on a from-source install of eth-account. Released versions are broken with:

>>> from eth_account import Account
>>> Account.enable_unaudited_hdwallet_features()
>>> Account.create_with_mnemonic()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jcarver/code/eth-account/tmp/non-develop-venv/lib/python3.6/site-packages/eth_utils/decorators.py", line 20, in _wrapper
    return self.method(objtype, *args, **kwargs)
  File "/home/jcarver/code/eth-account/tmp/non-develop-venv/lib/python3.6/site-packages/eth_account/account.py", line 333, in create_with_mnemonic
    mnemonic = generate_mnemonic(num_words, language)
  File "/home/jcarver/code/eth-account/tmp/non-develop-venv/lib/python3.6/site-packages/eth_account/hdaccount/__init__.py", line 16, in generate_mnemonic
    return Mnemonic(lang).generate(num_words)
  File "/home/jcarver/code/eth-account/tmp/non-develop-venv/lib/python3.6/site-packages/eth_account/hdaccount/mnemonic.py", line 79, in __init__
    f'Invalid language choice "{language}", must be one of {languages}'
eth_utils.exceptions.ValidationError: Invalid language choice "english", must be one of []

How can it be fixed?

1. Make sure that tests run against a "clean-room" installed version of eth-account (at least for the mnemonic tests)
2. Package up the language files to include in the release

What was wrong?

No support for EIP712 style signed messages: https://github.com/ethereum/EIPs/blob/master/EIPS/eip-712.md

How can it be fixed?

Implement a low level signing API for creating and verifying EIP712 style signatures with appropriate primatives for wiring up to a higher level integration into the Account class.

Account.encrypt() doesn't provide means to set the iterations (work-factor) parameter of the eth_keyfile.keyfile.create_keyfile_json() method.
It would be useful if that was somewhat possible without monkey patching the call.
My use case is I'm running Python-based "Ethereum clients" on smart-phones (e.g. EtherollApp, PyWallet) and it has less resources so account creation with default pbkdf2 params may take ages.
I was initially using pyethereum where it was somewhat possible overriding PBKDF2_CONSTANTS['c'], but I'm in the process of migrating it since it's deprecated.

They way around I'm currently using with eth-account is to monkey patch create_keyfile_json() call.

import eth_account
from eth_keyfile import create_keyfile_json

account = eth_account.Account.create()
eth_account.account.create_keyfile_json = partial(
    create_keyfile_json, iterations=1)
encrypted = eth_account.Account.encrypt(account.privateKey, "password")

I know setting iterations=1 is terribly wrong but it's for demo-purpose. It could be useful for speeding up unit testing too.

Anyway, if the use-case of giving access to this param makes sense, I could make a pull request.

What was wrong?

Our most recent pattern for tools that are not guaranteed to be API stable is to put them in a _utils module. internal is used for that purpose in eth-account, but it would be nice to use the new pattern.

How can it be fixed?

Rename internal -> _utils, and update all the appropriate imports.

web3.py uses pbkdf2(dklen:32, c:1000000, prf: hmac-sha256).
these are pretty low defaults
should def at least use scrypt as a baseline
hmac-sha256 is much easier to bruteforce, by like 3-4 orders of magnitude
the parameters don't matter i think the KDF func/lib should be able to accommodate but probably best to make them match the geth defaults
which are more secure. for some comparison an iteration of the default SCRYPT parameters in geth takes 200-400ms but an iteration of the HMAC KDF takes less than 1ms, on GPUs SCRYPT with r=8 destroys performance because of the high memory bandwidth > requirement, so even on super high end GPUs you can barely get 20-30h/s where as with the HMAC KDF you can get 100k-1m h/s a second.

• geth
• parity
• X web3.js (if text non-ascii)
• metamask?

See quirk in #7

Otherwise, it should be similar to getting the doctests running in web3.py.

I'd like to submit PR if you'll accept it ;D

There is a typo in

eth_account/_utils/transactions.py

on lines 128, and 133. I'm not submitting a PR because I'm not sure if these may break something downstream.

I guess #67 implicitly adds support for tuple types in the typed data signing, maybe more that weren't previously supported. Probably worth adding some tests to prove it and make sure we don't have regressions.

What was wrong?

We currently don't seem to validate types when using encode_structured_data (i.e. bytes32, uint8, uint256 sizes).

Relevant issues that sparked the creation of this issue: #90, #91

How can it be fixed?

• Investigate if this validation exists and if it is needed anywhere else in the code base.

Closing edit: Indeed it does exist, just not in the "validation" part of things. Validation seems to validate that the fields of the body are valid fields but value validation happens during the actual hashing.

What was wrong?

Missing and/or wrong type definitions.

How can it be fixed?

All of the strictness checks in mypy.ini should be set to True, like:
https://github.com/ethereum/ethereum-python-project-template/blob/582e9aa475bb56befb9a56188d2d33d176442b19/mypy.ini

There is a decent amount of work to do to add type annotations. There are a few wrong definitions too. Fix those up, and then the stricter checks will enforce it going forward.

What was wrong?

I cannot encode bytes32 in structured data. I am attempting to pass a struct containing a variable I want encoded as bytes32 to eth_account.messages.encode_structured_data. I am passing this struct to encode_structured_data as a Python dictionary. Including my bytes32 variable as a bytestring causes a JSON serialization error (eth_account/messages.py:143). Passing this variable as a hex string throws a TypeError (eth_account/_utils/structured_data/hashing.py:250).

Code that produced the error

from eth_account.messages import (
    encode_structured_data,
)
from eth_utils import (
    keccak,
)

message = {
	"types": {
		"EIP712Domain": [
			{"name": "name", "type": "string"},
			{"name": "version", "type": "string"},
			{"name": "chainId", "type": "uint256"},
			{"name": "verifyingContract", "type": "address"}
		],
		"coolStruct": [
			{"name": "coolBytes", "type": "bytes32"}
		]
	},
	"primaryType": "coolStruct",
	"domain": {
		"name": "coolContract",
		"version": "1",
		"chainId": 1,
		"verifyingContract": "0xCcCCccccCCCCcCCCCCCcCcCccCcCCCcCcccccccC"
	},
	"message": {
		"coolBytes": keccak(b''), # Alternativley keccak(b'').hex()
	}
}

def test_hashed_structured_data():
    structured_msg = encode_structured_data(message)

Full error output

eth_account/messages.py:143: in encode_structured_data
    message_string = json.dumps(primitive)
TypeError: Object of type bytes is not JSON serializable

TypeError: Value of `coolBytes` (c5d2460186f7233c927e7db2dcc703c0e500b653ca82273b7bfad8045d85a470) in the struct `coolStruct` is of the type `<class 'str'>`, but expected bytes32 value

Expected Result

It can definitely be done and should equal something like this:

    domain_type_hash = keccak(text='EIP712Domain(string name,string version,uint256 chainId,address verifyingContract)')
    name = message['domain']['name']
    version = message['domain']['version']
    chain_id = message['domain']['chainId']
    verifying_contract = message['domain']['verifyingContract']
    domain_instance = encode_abi(
      ['bytes32', 'bytes32', 'bytes32', 'uint256', 'address'],
      [domain_type_hash, keccak(text=name), keccak(text=version), chain_id, verifying_contract],
    )
    domain_hash = keccak(domain_instance)

    cool_struct_type_hash = keccak(text='coolStruct(bytes32 coolBytes)')
    cool_bytes = message['message']['coolBytes']
    cool_struct_instance = encode_abi(
      ['bytes32', 'bytes32'],
      [cool_struct_type_hash, cool_bytes]
    )
    cool_struct_hash = keccak(cool_struct_instance)

    preamble = b'\x19'
    version = b'\x01'

    message_hash = keccak(preamble + version + domain_hash + cool_struct_hash)
    print(message_hash.hex()) # 6cb68ad2b02b4a7f086020d9c23dc1d9f3a56d60e9441b26b6fdd90e2518277a

How can it be fixed?

I'm posting an issue first before attempting to fix to ask yall how best you think this can be solved? Why do we need to json.dumps? Seems like thats a blocker since bytes can't be easily serialized into JSON.

PR #91

originally based on branch #34 I created ledgertools, a fully functional suite to interact with ledger. to merge back into eth-account some more work is needed. this library focuses on some extra security measures for signing which might be useful to add to eth-account.

https://github.com/vegaswap/ledgertools

This bit of validation:

where the type of eip712Domain is a struct named EIP712Domain with one or more of the below fields. Protocol designers only need to include the fields that make sense for their signing domain. Unused fields are left out of the struct type.

Account.signTransaction is incorrectly accepting transaction dicts with to field set to a non-checksummed address.

Fix that, and look for other places this validation may be missing.

See https://gist.github.com/holiman/6f57cb5b082b6712b9e8c19fe48ca960#gistcomment-2332077

What was wrong?

If addresses are produced programatically, it is reasonable to skip the checksum check. There is no way to do this currently.

How can it be fixed?

Accept native python bytes as a to address value, when signing a transaction. It must still be 20 bytes of course.

What was wrong?

Currently eth-account is still supporting python 3.5. But to replace utils method is_valid_abi_type with ABI's method is_encodable_type, it is possible only when the support of python 3.5 is removed by eth-account, because eth-abi currently doesn't support python 3.5

How can it be fixed?

Replace the function, when the support for python 3.5 is dropped in eth-account.