Giter Club home page Giter Club logo

eth-tester's Introduction

eth-tester

Join the conversation on Discord Build Status PyPI version Python versions

Tools for testing Ethereum applications

Read more in the documentation below. View the change log.

Quick Start

python -m pip install eth-tester
>>> from eth_tester import EthereumTester
>>> t = EthereumTester()
>>> t.get_accounts()
('0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf',
 '0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF',
 '0x6813Eb9362372EEF6200f3b1dbC3f819671cBA69',
 '0x1efF47bc3a10a45D4B230B5d10E37751FE6AA718',
 '0xe1AB8145F7E55DC933d51a18c793F901A3A0b276',
 '0xE57bFE9F44b819898F47BF37E5AF72a0783e1141',
 '0xd41c057fd1c78805AAC12B0A94a405c0461A6FBb',
 '0xF1F6619B38A98d6De0800F1DefC0a6399eB6d30C',
 '0xF7Edc8FA1eCc32967F827C9043FcAe6ba73afA5c',
 '0x4CCeBa2d7D2B4fdcE4304d3e09a1fea9fbEb1528')

>>> t.get_balance('0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf')
1000000000000000000000000

>>> t.send_transaction({
...     'from': '0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf',
...     'to': '0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF',
...     'gas': 30000,
...     'value': 1,
...     'max_fee_per_gas': 1000000000,
...     'max_priority_fee_per_gas': 1000000000,
...     'chain_id': 131277322940537,
...     'access_list': (
...         {
...             'address': '0xde0b295669a9fd93d5f28d9ec85e40f4cb697bae',
...             'storage_keys': (
...                 '0x0000000000000000000000000000000000000000000000000000000000000003',
...                 '0x0000000000000000000000000000000000000000000000000000000000000007',
...             )
...         },
...         {
...             'address': '0xbb9bc244d798123fde783fcc1c72d3bb8c189413',
...             'storage_keys': ()
...         },
...     )
... })
'0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109'

>>> t.get_transaction_by_hash('0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109')
{'type': '0x2',
 'hash': '0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109',
 'nonce': 0,
 'block_hash': '0x28b95514984b0abbd91d88f1a542eaeeb810c24e0234e09891b7d6b3f94f47ed',
 'block_number': 1,
 'transaction_index': 0,
 'from': '0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf',
 'to': '0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF',
 'value': 1,
 'gas': 30000,
 'data': '0x',
 'r': 60071646517429056848243893841817235885102606421189844318110381014348740252962,
 's': 55731679314783756278323646144996847004593793888590884914350251538533006990589,
 'v': 0,
 'chain_id': 131277322940537,
 'max_fee_per_gas': 1000000000,
 'max_priority_fee_per_gas': 1000000000,
 'access_list': ({'address': '0xde0B295669a9FD93d5F28D9Ec85E40f4cb697BAe',
   'storage_keys': ('0x0000000000000000000000000000000000000000000000000000000000000003',
    '0x0000000000000000000000000000000000000000000000000000000000000007')},
  {'address': '0xBB9bc244D798123fDe783fCc1C72d3Bb8C189413',
   'storage_keys': ()}),
 'gas_price': 1000000000}


>>> t.get_transaction_receipt('0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109')
{'transaction_hash': '0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109',
 'transaction_index': 0,
 'block_number': 1,
 'block_hash': '0x28b95514984b0abbd91d88f1a542eaeeb810c24e0234e09891b7d6b3f94f47ed',
 'cumulative_gas_used': 29600,
 'gas_used': 29600,
 'effective_gas_price': 1000000000,
 'contract_address': None,
 'logs': (),
 'type': '0x2',
 'status': 1}

Documentation

Input and output data formats

The ethereum tester library strictly enforces the following input formats and types.

  • Hexadecimal values must be text (not byte) strings. The 0x prefix is optional.
  • Any address which contains mixed-case alpha characters will be validated as a checksummed address as specified by EIP-55
  • 32-byte hashes must be hexadecimal encoded.
  • Numeric values must be in their integer representation.

Similarly, ethereum tester ensures that return values conform to similar rules.

  • 32-byte hashes will be returned in their hexadecimal encoded representation.
  • Addresses will be returned in their hexadecimal representation and EIP55 checksummed.
  • Numeric values will be returned as integers.

Block Numbers

Any block_number parameter will accept the following string values.

  • 'latest': for the latest mined block.
  • 'pending': for the current un-mined block.
  • 'earliest': for the genesis block.
  • 'safe': for the last block that has passed 2/3 of attestations post-merge.
  • 'finalized': for the last finalized block post-merge.

Note: These must be text strings (not byte stringS)

eth_tester.EthereumTester

API

Instantiation

  • eth_tester.EthereumTester(backend=None, validator=None, normalizer=None, auto_mine_transactions=True, fork_blocks=None)

The EthereumTester object is the sole API entrypoint. Instantiation of this object accepts the following parameters.

>>> from eth_tester import EthereumTester
>>> t = EthereumTester()
>>> t
<eth_tester.main.EthereumTester at 0x102255710>

Fork Rules

Ethereum tester uses the Paris (PoS) fork rules, starting at block 0.

Time Travel

The chain can only time travel forward in time.

EthereumTester.time_travel(timestamp)

The timestamp must be an integer, strictly greater than the current timestamp of the latest block.

Note: Time traveling will result in a new block being mined.

Mining

Manually mining blocks can be done with the following API. The coinbase parameter of these methods must be a hexadecimal encoded address.

EthereumTester.mine_blocks(num_blocks=1, coinbase=ZERO_ADDRESS)

Mines num_blocks new blocks, returning an iterable of the newly mined block hashes.

EthereumTester.mine_block(coinbase=ZERO_ADDRESS)

Mines a single new block, returning the mined block's hash.

Auto-mining transactions

By default, all transactions are mined immediately. This means that each transaction you send will result in a new block being mined, and that all blocks will only ever have at most a single transaction. This behavior can be controlled with the following methods.

EthereumTester.enable_auto_mine_transactions()

Turns on auto-mining of transactions.

EthereumTester.disable_auto_mine_transactions()

Turns off auto-mining of transactions.

Accounts

The following API can be used to interact with account data. The account parameter in these methods must be a hexadecimal encode address.

EthereumTester.get_accounts()

Returns an iterable of the accounts that the tester knows about. All accounts in this list will be EIP55 checksummed.

>>> t.get_accounts()
('0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1',
 '0x7d577a597B2742b498Cb5Cf0C26cDCD726d39E6e',
 ...
 '0x90F0B1EBbbA1C1936aFF7AAf20a7878FF9e04B6c')

EthereumTester.add_account(private_key, password=None)

Adds a new account for the given private key. Returns the hex encoded address of the added account.

>>> t.add_account('0x58d23b55bc9cdce1f18c2500f40ff4ab7245df9a89505e9b1fa4851f623d241d')
'0xdc544d1aa88ff8bbd2f2aec754b1f1e99e1812fd'

By default, added accounts are unlocked and do not have a password. If you would like to add an account which has a password, you can supply the password as the second parameter.

>>> t.add_account('0x58d23b55bc9cdce1f18c2500f40ff4ab7245df9a89505e9b1fa4851f623d241d', 'my-secret')
'0xdc544d1aa88ff8bbd2f2aec754b1f1e99e1812fd'

EthereumTester.unlock_account(account, password, unlock_seconds=None)

Unlocks the given account if the provided password matches.

Raises a ValidationError if:

  • The account is not known.
  • The password does not match.
  • The account was created without a password.
>>> t.unlock_account('0xdc544d1aa88ff8bbd2f2aec754b1f1e99e1812fd', 'my-secret')

By default, accounts will be unlocked indefinitely. You can however unlock an account for a specified amount of time by providing the desired duration in seconds.

# unlock for 1 hour.
>>> t.unlock_account('0xdc544d1aa88ff8bbd2f2aec754b1f1e99e1812fd', 'my-secret', 60 * 60)

EthereumTester.lock_account(account)

Locks the provided account.

Raises a ValidationError if:

  • The account is not known
  • The account does not have a password.

EthereumTester.get_balance(account) -> integer

Returns the balance, in wei, for the provided account.

>>> t.get_balance('0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1')
1000004999999999999999999

EthereumTester.get_nonce(account) -> integer

Returns the nonce for the provided account.

>>> t.get_nonce('0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1')
1

EthereumTester.get_code(account) -> hex string

Returns the code for the given account.

>>> t.get_code('0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1')
"0x"

Blocks, Transactions, and Receipts

EthereumTester.get_transaction_by_hash(transaction_hash) -> transaction-object

Returns the transaction for the given hash, raising a TransactionNotFound exception if the transaction cannot be found.

>>> t.get_transaction_by_hash('0x21ae665f707e12a5f1bb13ef8c706b65cc5accfd03e7067ce683d831f51122e6')
{'type': '0x2',
 'hash': '0x21ae665f707e12a5f1bb13ef8c706b65cc5accfd03e7067ce683d831f51122e6',
 'nonce': 0,
 'block_hash': '0x810731efeb7498fc0ac3bc7c72a71571b672c9fdbfbfd8b435f483e368e8ef7e',
 'block_number': 1,
 'transaction_index': 0,
 'from': '0x2B5AD5c4795c026514f8317c7a215E218DcCD6cF',
 'to': '0x7E5F4552091A69125d5DfCb7b8C2659029395Bdf',
 'value': 1337,
 'gas': 21000,
 'data': '0x',
 'r': 1713666669454033023988006960017431058214051587080823768269189498559514600280,
 's': 32003859822305799628524852194521134173285969678963273753063458725692016415033,
 'v': 0,
 'chain_id': 131277322940537,
 'max_fee_per_gas': 2000000000,
 'max_priority_fee_per_gas': 500000000,
 'access_list': (),
 'gas_price': 1375000000}

Note: For unmined transaction, transaction_index, block_number and block_hash will all be None.

EthereumTester.get_block_by_number(block_number, full_transactions=False) -> block-object

Returns the block for the given block_number. See block numbers for named block numbers you can use. If full_transactions is truthy, then the transactions array will be populated with full transaction objects as opposed to their hashes.

Raises BlockNotFound if a block for the given number cannot be found.

>>> t.get_block_by_number(1)
{'number': 1,
 'hash': '0xd481955268d1f3db58ee61685a899a35e33e8fd35b9cc0812f85b9f06757140e',
 'parent_hash': '0x5be984ab842071903ee443a5dee92603bef42de35b4e10928e753f7e88a7163a',
 'nonce': '0x0000000000000000',
 'sha3_uncles': '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347',
 'logs_bloom': 0,
 'transactions_root': '0xef1e11d99f7db22fd93c6a10d44753d4a93e9f6ecb2f1e5030a0a91f1d3b07ac',
 'receipts_root': '0x611e48488cf80b4c31f01ad45b6ebea533a68255a6d0240d434d9366a3582010',
 'state_root': '0x9ce568dcaa6f130d733b333304f2c26a19334ed328a7eb9bb31707306381ba65',
 'coinbase': '0x0000000000000000000000000000000000000000',
 'difficulty': 0,
 'total_difficulty': 0,
 'mix_hash': '0x0000000000000000000000000000000000000000000000000000000000000000',
 'size': 751,
 'extra_data': '0x0000000000000000000000000000000000000000000000000000000000000000',
 'gas_limit': 3141592,
 'gas_used': 29600,
 'timestamp': 1633669276,
 'transactions': ('0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109',),
 'uncles': (),
 'base_fee_per_gas': 875000000}

EthereumTester.get_block_by_hash(block_hash, full_transactions=True) -> block-object

Returns the block for the given block_hash. The full_transactions parameter behaves the same as in get_block_by_number.

Raises BlockNotFound if a block for the given hash cannot be found.

>>> t.get_block_by_hash('0x0f50c8ea0f67ce0b7bff51ae866159edc443bde87de2ab26010a15b777244ddd')
{'number': 1,
 'hash': '0xd481955268d1f3db58ee61685a899a35e33e8fd35b9cc0812f85b9f06757140e',
 'parent_hash': '0x5be984ab842071903ee443a5dee92603bef42de35b4e10928e753f7e88a7163a',
 'nonce': '0x0000000000000000',
 'sha3_uncles': '0x1dcc4de8dec75d7aab85b567b6ccd41ad312451b948a7413f0a142fd40d49347',
 'logs_bloom': 0,
 'transactions_root': '0xef1e11d99f7db22fd93c6a10d44753d4a93e9f6ecb2f1e5030a0a91f1d3b07ac',
 'receipts_root': '0x611e48488cf80b4c31f01ad45b6ebea533a68255a6d0240d434d9366a3582010',
 'state_root': '0x9ce568dcaa6f130d733b333304f2c26a19334ed328a7eb9bb31707306381ba65',
 'coinbase': '0x0000000000000000000000000000000000000000',
 'difficulty': 0,
 'total_difficulty': 0,
 'mix_hash': '0x0000000000000000000000000000000000000000000000000000000000000000',
 'size': 751,
 'extra_data': '0x0000000000000000000000000000000000000000000000000000000000000000',
 'gas_limit': 3141592,
 'gas_used': 29600,
 'timestamp': 1633669276,
 'transactions': ('0xc20b90af87bc65c3d748cf0a1fa54f3a86ffc94348e0fd91a70f1c5ba6ef4109',),
 'uncles': (),
 'base_fee_per_gas': 875000000}

EthereumTester.get_transaction_receipt(transaction_hash)

Returns the receipt for the given transaction_hash, raising TransactionNotFound if no transaction can be found for the given hash.

>>> t.get_transaction_receipt('0x9a7cc8b7accf54ecb1901bf4d0178f28ca457bb9f9c245692c0ca8fabef08d3b')
 {'block_hash': '0x878f779d8bb25b25fb78fc16b8d64d70a5961310ef1689571aec632e9424290c',
 'block_number': 2,
 'contract_address': None,
 'cumulative_gas_used': 23154,
 'gas_used': 23154,
 'logs': ({'address': '0xd6F084Ee15E38c4f7e091f8DD0FE6Fe4a0E203Ef',
   'block_hash': '0x878f779d8bb25b25fb78fc16b8d64d70a5961310ef1689571aec632e9424290c',
   'block_number': 2,
   'data': '0x',
   'log_index': 0,
   'topics': (
    '0xf70fe689e290d8ce2b2a388ac28db36fbb0e16a6d89c6804c461f65a1b40bb15',
    '0x0000000000000000000000000000000000000000000000000000000000003039'),
   'transaction_hash': '0x9a7cc8b7accf54ecb1901bf4d0178f28ca457bb9f9c245692c0ca8fabef08d3b',
   'transaction_index': 0,
   'type': 'mined'},),
 'transaction_hash': '0x9a7cc8b7accf54ecb1901bf4d0178f28ca457bb9f9c245692c0ca8fabef08d3b',
 'transaction_index': 0}
  • Receipts for unmined transactions will have all of block_hash, block_number and transaction_index set to None.
  • Receipts for transactions which create a contract will have the created contract address in the contract_address field.

Transaction Sending

A transaction is a formatted as a dictionary with the following keys and values.

  • from: The address of the account sending the transaction (hexadecimal string).
  • to: The address of the account the transaction is being sent to. Empty string should be used to trigger contract creation (hexadecimal string).
  • gas: Sets the gas limit for transaction execution (integer).
  • value: The amount of ether in wei that should be sent with the transaction (integer).
  • data: The data for the transaction (hexadecimal string).
  • chain_id: The integer id for the chain the transaction is meant to interact with.

In addition to the above, the following parameters are added based on the type of transaction being sent:

Legacy transactions

  • gas_price: Sets the price per unit of gas in wei that will be paid for transaction execution (integer).

Access list transactions (EIP-2930)

  • gas_price: Sets the price per unit of gas in wei that will be paid for transaction execution (integer).
  • access_list (optional): Specifies accounts and storage slots expected to be accessed, based on the transaction, in order to gain a discount on the gas for those executions (see quickstart example for usage).

Dynamic fee transactions (EIP-1559)

  • max_fee_per_gas: Sets the maximum fee per unit of gas in wei that will be paid for transaction execution (integer).
  • max_priority_fee_per_gas: Sets the fee per unit of gas in wei that is sent to the coinbase address as an incentive for including the transaction (integer).
  • access_list (optional): Specifies accounts and storage slots expected to be accessed, based on the transaction, in order to gain a discount on the gas for those executions (see quickstart example for usage).

Methods

EthereumTester.send_transaction(transaction) -> transaction_hash

Sends the provided transaction object, returning the transaction_hash for the sent transaction.

EthereumTester.call(transaction, block_number='latest')

Executes the provided transaction object at the evm state from the block denoted by the block_number parameter, returning the resulting bytes return value from the evm.

EthereumTester.estimate_gas(transaction)

Executes the provided transaction object, measuring and returning the gas consumption.

EthereumTester.get_fee_history(block_count=1, newest_block='latest', reward_percentiles=[])

Return the historical gas information for the number of blocks specified as the block_count starting from newest_block. Note that specifying reward_percentiles has no effect on the response and so reward will always return an empty list.

Logs and Filters

EthereumTester.create_block_filter() -> integer

Creates a new filter for newly mined blocks. Returns the filter_id which can be used to retrieve the block hashes for the mined blocks.

>>> filter_id = t.create_block_filter()
>>> filter_id = t.create_block_filter()
>>> t.mine_blocks(3)
>>> t.get_only_filter_changes(filter_id)
('0x07004287f82c1a7ab15d7b8baa03ac14d7e9167ab74e47e1dc4bd2213dd18431',
 '0x5e3222c506585e1202da08c7231afdc5e472c777c245b822f44f141d335c744a',
 '0x4051c3ba3dcca95da5db1be38e44f5b47fd1a855ba522123e3254fe3f8e271ea')
>>> t.mine_blocks(2)
>>> t.get_only_filter_changes(filter_id)
('0x6649c3a7cb3c7ede3a4fd10ae9dd63775eccdafe39ace5f5a9ae81d360089fba',
 '0x04890a08bca0ed2f1496eb29c5dc7aa66014c85377c6d9d9c2c315f85204b39c')
>>> t.get_all_filter_logs(filter_id)
('0x07004287f82c1a7ab15d7b8baa03ac14d7e9167ab74e47e1dc4bd2213dd18431',
 '0x5e3222c506585e1202da08c7231afdc5e472c777c245b822f44f141d335c744a',
 '0x4051c3ba3dcca95da5db1be38e44f5b47fd1a855ba522123e3254fe3f8e271ea',
 '0x6649c3a7cb3c7ede3a4fd10ae9dd63775eccdafe39ace5f5a9ae81d360089fba',
 '0x04890a08bca0ed2f1496eb29c5dc7aa66014c85377c6d9d9c2c315f85204b39c')

EthereumTester.create_pending_transaction_filter() -> integer

Creates a new filter for pending transactions. Returns the filter_id which can be used to retrieve the transaction hashes for the pending transactions.

>>> filter_id = t.create_pending_transaction_filter()
>>> t.send_transaction({...})
'0x07f20bf9586e373ac914a40e99119c4932bee343d89ba852ccfc9af1fd541566'
>>> t.send_transaction({...})
'0xff85f7751d132b66c03e548e736f870797b0f24f3ed41dfe5fc628eb2cbc3505'
>>> t.get_only_filter_changes(filter_id)
('0x07f20bf9586e373ac914a40e99119c4932bee343d89ba852ccfc9af1fd541566',
 '0xff85f7751d132b66c03e548e736f870797b0f24f3ed41dfe5fc628eb2cbc3505')
>>> t.send_transaction({...})
'0xb07801f7e8b1cfa52b64271fa2673c4b8d64cc21cdbc5fde51d5858c94c2d26a'
>>> t.get_only_filter_changes(filter_id)
('0xb07801f7e8b1cfa52b64271fa2673c4b8d64cc21cdbc5fde51d5858c94c2d26a',)
>>> t.get_all_filter_logs(filter_id)
('0x07f20bf9586e373ac914a40e99119c4932bee343d89ba852ccfc9af1fd541566',
 '0xff85f7751d132b66c03e548e736f870797b0f24f3ed41dfe5fc628eb2cbc3505',
 '0xb07801f7e8b1cfa52b64271fa2673c4b8d64cc21cdbc5fde51d5858c94c2d26a')

EthereumTester.create_log_filter(from_block=None, to_block=None, address=None, topics=None) -> integer

Creates a new filter for logs produced by transactions. The parameters for this function can be used to filter the log entries.

>>> filter_id = t.create_log_filter()
>>> t.send_transaction({...})  # something that produces a log entry
'0x728bf75fc7d23845f328d2223df7fe9cafc6e7d23792457b625d5b60d2b22b7c'
>>> t.send_transaction({...})  # something that produces a log entry
'0x63f5b381ffd09940ce22c45a3f4e163bd743851cb6b4f43771fbf0b3c14b2f8a'
>>> t.get_only_filter_changes(filter_id)
({'address': '0xd6F084Ee15E38c4f7e091f8DD0FE6Fe4a0E203Ef',
  'block_hash': '0x68c0f318388003b652eae334efbed8bd345c469bd0ca77469183fc9693c23e13',
  'block_number': 11,
  'data': '0x',
  'log_index': 0,
  'topics': ('0xf70fe689e290d8ce2b2a388ac28db36fbb0e16a6d89c6804c461f65a1b40bb15',
   '0x0000000000000000000000000000000000000000000000000000000000003039'),
  'transaction_hash': '0x728bf75fc7d23845f328d2223df7fe9cafc6e7d23792457b625d5b60d2b22b7c',
  'transaction_index': 0,
  'type': 'mined'},
 {'address': '0xd6F084Ee15E38c4f7e091f8DD0FE6Fe4a0E203Ef',
  'block_hash': '0x07d7e46be6f9ba53ecd4323fb99ec656e652c4b14f4b8e8a244ee7f997464725',
  'block_number': 12,
  'data': '0x',
  'log_index': 0,
  'topics': ('0xf70fe689e290d8ce2b2a388ac28db36fbb0e16a6d89c6804c461f65a1b40bb15',
   '0x0000000000000000000000000000000000000000000000000000000000010932'),
  'transaction_hash': '0x63f5b381ffd09940ce22c45a3f4e163bd743851cb6b4f43771fbf0b3c14b2f8a',
  'transaction_index': 0,
  'type': 'mined'})

EthereumTester.delete_filter(filter_id)

Removes the filter for the provided filter_id. If no filter is found for the given filter_id, raises FilterNotFound.

EthereumTester.get_only_filter_changes(filter_id) -> transaction_hash or block_hash or log_entry

Returns all new values for the provided filter_id that have not previously been returned through this API. Raises FilterNotFound if no filter is found for the given filter_id.

EthereumTester.get_all_filter_logs(filter_id) -> transaction_hash or block_hash or log_entry

Returns all values for the provided filter_id. Raises FilterNotFound if no filter is found for the given filter_id.

Snapshots and Resetting

EthereumTester.take_snapshot() -> snapshot_id

Takes a snapshot of the current chain state and returns the snapshot id.

EthereumTester.revert_to_snapshot(snapshot_id)

Reverts the chain to the chain state associated with the given snapshot_id. Raises SnapshotNotFound if no snapshot is known for the given id.

Errors and Exceptions

eth_tester.exceptions.TransactionNotFound

Raised in cases where a transaction cannot be found for the provided transaction hash.

eth_tester.exceptions.BlockNotFound

Raised in cases where a block cannot be found for either a provided number or hash.

eth_tester.exceptions.FilterNotFound

Raised in cases where a filter cannot be found for the provided filter id.

eth_tester.exceptions.SnapshotNotFound

Raised in cases where a snapshot cannot be found for the provided snapshot id.

Backends

Ethereum tester is written using a pluggable backend system.

Backend Dependencies

Ethereum tester does not install any of the dependencies needed to use the various backends by default. You can however install ethereum tester with the necessary dependencies using the following method.

$ python -m pip install eth-tester[<backend-name>]

You should replace <backend-name> with the name of the desired testing backend. Available backends are:

Selecting a Backend

You can select which backend in a few different ways.

The most direct way is to manually pass in the backend instance you wish to use.

>>> from eth_tester import EthereumTester, MockBackend
>>> t = EthereumTester(backend=MockBackend())

Ethereum tester also supports configuration using the environment variable ETHEREUM_TESTER_CHAIN_BACKEND. This should be set to the import path for the backend class you wish to use.

Available Backends

Ethereum tester can be used with the following backends.

  • MockBackend
  • PyEVM (experimental)

MockBackend

This backend has limited functionality. It cannot perform any VM computations. It mocks out all of the objects and interactions.

>>> from eth_tester import EthereumTester, MockBackend
>>> t = EthereumTester(MockBackend())

PyEVM (experimental)

WARNING Py-EVM is experimental and should not be relied on for mission critical testing at this stage.

Uses the experimental Py-EVM library.

>>> from eth_tester import EthereumTester, PyEVMBackend
>>> t = EthereumTester(PyEVMBackend())

PyEVM Genesis Parameters and State

If you need to specify custom genesis parameters and state, you can build your own parameters dict to use instead of the default when initializing a backend. Only default values can be overridden or a ValueError will be raised.

# Default Genesis Parameters

default_genesis_params = {
    "coinbase": GENESIS_COINBASE,
    "difficulty": GENESIS_DIFFICULTY,
    "extra_data": GENESIS_EXTRA_DATA,
    "gas_limit": GENESIS_GAS_LIMIT,
    "mix_hash": GENESIS_MIX_HASH,
    "nonce": GENESIS_NONCE,
    "receipt_root": BLANK_ROOT_HASH,
    "timestamp": int(time.time()),
    "transaction_root": BLANK_ROOT_HASH,
}

To generate a genesis parameters dict with an overridden parameters, pass a genesis_overrides dict
to PyEVM.generate_genesis_params.

>>> from eth_tester import PyEVMBackend, EthereumTester

>>> genesis_overrides = {'gas_limit': 4500000}
>>> custom_genesis_params = PyEVMBackend.generate_genesis_params(overrides=genesis_overrides)

# Generates the following `dict`:

# custom_genesis_params = {
#     "coinbase": GENESIS_COINBASE,
#     "difficulty": GENESIS_DIFFICULTY,
#     "extra_data": GENESIS_EXTRA_DATA,
#     "gas_limit": 4500000    # <<< Overridden Value <<<
#     "mix_hash": GENESIS_MIX_HASH,
#     "nonce": GENESIS_NONCE,
#     "receipt_root": BLANK_ROOT_HASH,
#     "timestamp": int(time.time()),
#     "transaction_root": BLANK_ROOT_HASH,
# }

Then pass the generated custom_genesis_params dict to the backend's __init__

>>> from eth_tester import PyEVMBackend, EthereumTester
>>> pyevm_backend = PyEVMBackend(genesis_parameters=custom_genesis_params)
>>> t = EthereumTester(backend=pyevm_backend)

Similarly to genesis_parameters, override the genesis state by passing in an overrides dict to PyEVMBackend.generate_genesis_state. Optionally, provide num_accounts to set the number of accounts.

For more control on which accounts the backend generates, use the from_mnemonic() classmethod. Give it a mnemonic (and optionally the number of accounts) and it will use that information to generate the accounts. Optionally, provide a genesis_state_overrides dict to adjust the genesis_state.

>>> from eth_tester import PyEVMBackend, EthereumTester
>>> from eth_utils import to_wei
>>> from hexbytes import HexBytes
>>>
>>> pyevm_backend = PyEVMBackend.from_mnemonic(
>>>    'test test test test test test test test test test test junk',
>>>    genesis_state_overrides={'balance': to_wei(1000000, 'ether')}
>>> )
>>> t = EthereumTester(backend=pyevm_backend)
>>> print(t.get_accounts()[0])  # Outputs 0x1e59ce931B4CFea3fe4B875411e280e173cB7A9C
>>> print(t.get_balance('0x1e59ce931B4CFea3fe4B875411e280e173cB7A9C'))  # Outputs 1000000000000000000000000

NOTE: The same state is applied to all generated test accounts.

# Default Account Genesis State

default_account_state = {
    'balance': to_wei(1000000, 'ether'),
    'storage': {},
    'code': b'',
    'nonce': 0,
}

For Example, to create 3 test accounts, each with a balance of 100 ETH each:

>>> from eth_tester import EthereumTester, PyEVMBackend
>>>  from eth_utils import to_wei

>>> state_overrides = {'balance': to_wei(100, 'ether')}
>>> custom_genesis_state = PyEVMBackend.generate_genesis_state(overrides=state_overrides, num_accounts=3)

# Then pass the generated `custom_genesis_state` `dict` to the backend's `__init__`

>>> pyevm_backend = PyEVMBackend(genesis_state=custom_genesis_state)
>>> t = EthereumTester(backend=pyevm_backend)

Implementing Custom Backends

The base class eth_tester.backends.base.BaseChainBackend is the recommended base class to begin with if you wish to write your own backend.

Details on implementation are beyond the scope of this document.

Data Formats

Ethereum tester uses two formats for data.

  • The normal format is the data format the is expected as input arguments to all EthereumTester methods as well as the return types from all method calls.
  • The canonical format is the data format that is used internally by the backend class.

Ethereum tester enforces strict validation rules on these formats.

Canonical Formats

The canonical format is intended for low level handling by backends.

  • 32 byte hashes: bytes of length 32
  • Arbitrary length strings: bytes
  • Addresses: bytes of length 20
  • Integers: int
  • Array Types: tuple

Normal Formats

The normal format is intended for use by end users.

  • 32 byte hashes: 0x prefixed hexadecimal encoded text strings (not byte strings)
  • Arbitrary length strings: 0x prefixed hexadecimal encoded text strings (not byte strings)
  • Addresses: 0x prefixed and EIP55 checksummed hexadecimal encoded text strings (not byte strings)
  • Integers: int
  • Array Types: tuple

Normalization and Validation

Beware! Here there be dragons... This section of the documentation is only relevant if you intend to build tooling on top of this library.

The ethereum tester provides strong guarantees that backends can be swapped out seamlessly without effecting the data formats of both the input arguments and return values. This is accomplished using a two-step process of strict normalization and validation.

All inputs to the methods of the EthereumTester are first validated then normalized to a canonical format. Return values are put through this process as well, first validating the data returned by the backend, and then normalizing it from the canonical format to the normal form before being returned.

Normalization

The EthereumTester delegates normalization to whatever normalizer was passed in during instantiation. If no value was provided, the default normalizer will be used from eth_tester.normalization.default.DefaultNormalizer.

The specifics of this object are beyond the scope of this document.

Validation

The EthereumTester delegates validation to whatever validator was passed in during instantiation. If no value was provided, the default validator will be used from eth_tester.validation.default.DefaultValidator.

The specifics of this object are beyond the scope of this document.

Use with Web3.py

See the web3.py documentation for information on the EthereumTester provider which integrates with this library.

Developer Setup

If you would like to hack on eth-tester, please check out the Snake Charmers Tactical Manual for information on how we do:

  • Testing
  • Pull Requests
  • Documentation

We use pre-commit to maintain consistent code style. Once installed, it will run automatically with every commit. You can also run it manually with make lint. If you need to make a commit that skips the pre-commit checks, you can do so with git commit --no-verify.

Development Environment Setup

You can set up your dev environment with:

git clone [email protected]:ethereum/eth-tester.git
cd eth-tester
virtualenv -p python3 venv
. venv/bin/activate
python -m pip install -e ".[dev]"
pre-commit install

Release setup

To release a new version:

make release bump=$$VERSION_PART_TO_BUMP$$

How to bumpversion

The version format for this repo is {major}.{minor}.{patch} for stable, and {major}.{minor}.{patch}-{stage}.{devnum} for unstable (stage can be alpha or beta).

To issue the next version in line, specify which part to bump, like make release bump=minor or make release bump=devnum. This is typically done from the main branch, except when releasing a beta (in which case the beta is released from main, and the previous stable branch is released from said branch).

If you are in a beta version, make release bump=stage will switch to a stable.

To issue an unstable version when the current version is stable, specify the new version explicitly, like make release bump="--new-version 4.0.0-alpha.1 devnum"

eth-tester's People

Contributors

antazoey avatar berndbohmeier avatar brian-biometrix avatar cangqiaoyuzhuo avatar carver avatar cburgdorf avatar davesque avatar davidromanovizc avatar dylanjw avatar fselmo avatar fubuloubu avatar hwwhww avatar jacqueswww avatar jmyles avatar jstoxrocky avatar kclowes avatar kprasch avatar ldct avatar mhchia avatar monokh avatar nic619 avatar njgheorghita avatar onyb avatar pacrob avatar pcppcp avatar pipermerriam avatar reedsa avatar stefanmendoza avatar voith avatar wolovim avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

eth-tester's Issues

Make block gas limit configurable

  • Version: 0.1.0-beta.24
  • Python: 3.6
  • OS: osx

What was wrong?

Currently the block gas limit is hard coded:

GENESIS_GAS_LIMIT = 3141592

This fails some tests with larger contracts and should be made configurable. Also increasing the default might make sense, the mainnet blocks have been bigger than 6 million gas for some time.

How can it be fixed?

Make the block gas limit configurable and think about increasing the default value.

Mock storage variables in contracts?

I was curious as to whether it is theoretically possible to somehow add some functionality to the eth-tester library to mock state variables in contracts? I don't actually know too much about how data is stored in the EVM, and I'm coming from a Solidity mind-set so maybe I'm thinking about storage incorrectly. I was hoping for a reason why, if this is not possible, and maybe a hint of the right direction to start exploring this idea if it should be possible.

I would basically like functionality to do something like:

contract Example {
   function getVal() view public returns (uint256) {...}

   function doSomething() view public returns (bool) {
     uint256 x = getVal();
     require(x > 10);
     ...
  }
}
def test_contract(contract):
    EthereumTester.mock(contract, 'getVal').return_value = 11
    output = contract.functions.doSomething().call()
    assert output 

Handle pending transactions with a tx pool

  • Version: 0.1.0-beta.16
  • Python: all
  • OS: all

What was wrong?

Eth tester does not handle pending transactions correctly when disabling auto mining transactions:

Sending a transaction with the same nonce causes an invalid nonce error:

from eth_tester import (
    EthereumTester,
    PyEVMBackend,
)

backend = PyEVMBackend()
t = EthereumTester(backend=backend)
t.disable_auto_mine_transactions()
accounts = t.get_accounts()
tx = t.send_transaction({'from': accounts[0], 'to': accounts[1], 'gas': 21000, 'value': 1, 'nonce': 0})
tx2 = t.send_transaction({'from': accounts[0], 'to': accounts[1], 'gas': 21000, 'value': 1, 'nonce': 0})
>>> evm.exceptions.ValidationError: Invalid transaction nonce

How can it be fixed?

The issue is that send_transaction and send_raw_transaction immediately execute on the vm's state and thereby increment the valid nonce.

Things to consider for the fix:

  • A tx pool needs to be managed at the eth tester level
  • It should consider the rules for replacing transactions. e.g. transactions with existing nonce should overwrite #57
  • Will the tx pool work for every backend?
  • eth-tester exposes both send_transaction and send_raw_transaction both of which have different code paths. We need to ensure that we treat them the same as far as the tx pool is concerned.
  • eth-tester does not know about the outcome of a transaction (e.g. for figuring out what nonce a transaction will execute at). Could circumvent this by using get_transaction_by_hash after executing the transaction.

A very rudimentary fix:

if self.auto_mine_transactions:
    self.mine_block()
else:
    snapshot = self.take_snapshot()
    # If a pending transaction with the same nonce exists, overwrite it?
    self._pending_transactions.append(transaction)
    self.revert_to_snapshot(snapshot)

def enable_auto_mine_transactions(self):
    self.auto_mine_transactions = True
    return [self.send_transaction(tx) for tx in self._pending_transactions]

@carver @pipermerriam feel free to modify the issue to clarify things ๐Ÿ™‚

Creating a filter generates duplicate events in other filters

  • Version: 0.1.0b27
  • Python: 3.5
  • OS: linux

What was wrong?

When creating a new filter using create_log_filter(), _process_blocks_logs() is called to populate filter with past events.
This method however calls add() method for all existing filters:

    def _process_block_logs(self, block):
        for _, filter in self._log_filters.items():
            for transaction_hash in block['transactions']:
                receipt = self.get_transaction_receipt(transaction_hash)
                for log_entry in receipt['logs']:
                    raw_log_entry = self.normalizer.normalize_inbound_log_entry(log_entry)
                    filter.add(raw_log_entry)

How can it be fixed?

Filter::add() method should check if the log entry already exists in Filter::values before appending/processing it.

Allow call method to query state of past blocks

What was wrong?

Currently the eth-tester library doesn't support using the eth_call method to query the state of past blocks in the blockchain. Given that the new version of web3.py will support querying past blocks in the chain it makes sense to have that functionality available in eth-tester as well.

Overwritten transactions are not removed

  • Version: 0.1.0b15
  • Python: 3.5
  • OS: all

What was wrong?

When submitting a transaction that overwrites another transaction, the original transaction is not removed and calls to getTransaction() still return the old transaction instead of null.

In geth, the transaction would be removed and returned as None from getTransaction().

Example:

tester = EthereumTester(auto_mine_transactions=False)

web3 = Web3(EthereumTesterProvider(tester))

original_tx = web3.eth.sendTransaction({
    'from': '...',
    'to': '...',
    'value': 1,
    'gas': 23500,
    'gasPrice': 10,
    'nonce': 0
})

print(web3.eth.getTransaction(original_tx))

replacement_tx = web3.eth.sendTransaction({
    'from': '...',
    'to': '...',
    'value': 2,
    'gas': 23500,
    'gasPrice': 20,
    'nonce': 0
})

print(web3.eth.getTransaction(replacement_tx))
print(web3.eth.getTransaction(original_tx))  # Should be null but still returns transaction

How can it be fixed?

Unsure whether this is a backend problem or not but i imagine there needs to be some tx pool managing functionality added.

Accounts are not persisted when using LevelDB

  • py-evm Version: 0.20.a17
  • OS: osx
  • Python Version (python --version): 3.6.5
  • Environment (output of pip freeze):
    ..
    eth-abi==1.1.1
    eth-account==0.2.2
    eth-bloom==1.0.0
    eth-hash==0.1.2
    eth-keyfile==0.5.1
    eth-keys==0.2.0b3
    eth-rlp==0.1.2
    eth-tester==0.1.0b25
    eth-utils==1.0.3
    hexbytes==0.1.0
    ..
    py-ecc==1.4.2
    py-evm==0.2.0a17
    pycparser==2.18
    pycryptodome==3.6.1
    pyethash==0.1.27
    Pygments==2.2.0
    pypandoc==1.4
    pysha3==1.0.2
    pytest==3.5.1
    python-dateutil==2.7.3
    requests==2.18.4
    rlp==1.0.1
    ..

What was wrong?

When using LevelDB account keys are not persisted:

       # LevelDB will set all account balances equal to 0
       #db = ChainDB(LevelDB('./testleveldb.db'))

       # MemoryDB correctly persists account balances to 1000000000000000000000000
        db = ChainDB(MemoryDB())

        self.account_keys = get_default_account_keys()
        genesis_state = generate_genesis_state(self.account_keys)
        self.chain = chain_class.from_genesis(db, genesis_params, genesis_state)

Testing code to retrieve balances:

tester = EthereumTester(backend)
accounts = tester.get_accounts()
print(accounts)
print(list(tester.get_balance(account) for account in accounts))

Web3 - ValueError: RPC Endpoint has not been implemented: eth_estimateGas

Got a weird error with eth-tester:

$ py.test test_contract.py
test_contract.py:2: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
conftest.py:32: in deploy
    tx_hash = self.factory.deploy(args=args, **kwargs)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/contract.py:268: in deploy
    txn_hash = cls.web3.eth.sendTransaction(deploy_transaction)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/eth.py:212: in sendTransaction
    get_buffered_gas_estimate(self.web3, transaction),
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/utils/transactions.py:60: in get_buffered_gas_estimate
    gas_estimate = web3.eth.estimateGas(gas_estimate_transaction)
../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/eth.py:254: in estimateGas
    [transaction],
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <web3.manager.RequestManager object at 0x7f26a6b66208>, method = 'eth_estimateGas'
params = [{'data': '0x6060604052606060405190810160405280602481526020017f4461707044657673205368696c6c2d612d436f696e20576f726b736...baf09a012e63fb4bce3f65de49a48a38ebdd611cf6d4b7760029000000000000000000000000000000000000000000000000000000000000000a'}]

    def request_blocking(self, method, params):
        """
            Make a synchronous request using the provider
            """
        response = self._make_request(method, params)
    
        if "error" in response:
>           raise ValueError(response["error"])
E           ValueError: RPC Endpoint has not been implemented: eth_estimateGas

../../../../.pyenv/versions/3.6.4/envs/test/lib/python3.6/site-packages/web3/manager.py:102: ValueError

Setup:

  • Version: 0.1.0b18
  • Python: 3.6.4
  • OS: linux
$ pip freeze
attrdict==2.0.0
attrs==17.4.0
certifi==2018.1.18
chardet==3.0.4
cytoolz==0.9.0
eth-abi==1.0.0b0
eth-account==0.1.0a2
eth-hash==0.1.0a3
eth-keyfile==0.5.1
eth-keys==0.2.0b2
eth-rlp==0.1.0a2
eth-tester==0.1.0b18
eth-utils==1.0.0b1
hexbytes==0.1.0b0
idna==2.6
lru-dict==1.1.6
pluggy==0.6.0
py==1.5.2
pycryptodome==3.4.11
pysha3==1.0.2
pytest==3.4.1
requests==2.18.4
rlp==0.6.0
semantic-version==2.6.0
six==1.11.0
toolz==0.9.0
urllib3==1.22
web3==4.0.0b9

Replicate

'../contracts.json' is output of solc --combined-json abi,bin,bin-runtime ../contracts/*.sol

conftest.py:

import pytest

from web3 import Web3
from web3.providers.eth_tester import EthereumTesterProvider
from web3.contract import ImplicitContract
import json

w3 = Web3(EthereumTesterProvider())

@pytest.fixture
def a():
    return w3.personal.listAccounts

@pytest.fixture
def contracts():
    with open('../contracts.json', 'r') as f:
        contracts = json.loads(f.read())
    return contracts['contracts']

@pytest.fixture
def Contract(contracts):

    class Contract:
        def __init__(self, contract_name):
            interface = contracts[contract_name]
            self.abi = interface['abi']
            self.bin = interface['bin']
            self.runtime = interface['bin-runtime']
            self.factory = w3.eth.contract(abi=self.abi, bytecode=self.bin)

        def deploy(self, *args, **kwargs):
            tx_hash = self.factory.deploy(args=args, **kwargs)
            tx_receipt = w3.eth.getTransactionReceipt(tx_hash)
            address = tx_receipt['contractAddress']
            return w3.eth.contract(self.abi, address,
                    ContractFactoryClass=ImplicitContract)


    return Contract

test_contract.py:

def test_contract(Contract):
    Contract('contracts/ContractA.sol:ContractA').deploy()

nested topic arrays should be interpreted as match options for the topic at the nested array position

  • Version:0.1.0b29
  • Python: 3.6

What was wrong?

See review comment here: https://github.com/ethereum/web3.py/pull/976/files/bec2e4e082c2ac58fd38063dec5f5a22641bc3cc

To summarize: we were interpreting the nested arrays in topic lists as full variants of the topic list, rather than options for the topic at the nested array position.

How can it be fixed?

Several methods, validators etc. need to be updated to follow the spec here:

https://github.com/ethereum/wiki/wiki/JSON-RPC#a-note-on-specifying-topic-filters

Fix toolz and cytoolz code

What was wrong?

Right now all imports come from cytoolz. However, our setup.py module is setup to conditionally install either cytoolz or toolz depending on whether we're on Cython or pypy. This means that a pypy installation (which we aren't currently testing against) will fail because cytoolz won't be installed.

How can it be fixed?

Update all toolz/cytoolz imports to import from eth_utils.toolz.

Update gas limit

What was wrong?

Current default block gas limit is 3.14m gas. Mainchain gas limit has been around 8m for a while now, which means it is possible to deploy larger contracts than what is currently testable. Lowest testnet block gas limit is 7m gas.

How can it be fixed?

Until #129 is implemented to permanently solve the configuration issue, I would suggest that the arbitrarily-chosen default be at least 2x'd to 6.28m gas (2 Pi). This would ensure a more realistic block gas limit for testing, while retaining a sufficient margin of safety against the current limits (which can fluctuate slightly)

Snapshot state to a genesis file

What was wrong?

Inspired by #88 (comment)

People want to pick up chain state from somewhere they left off in a test chain. It would be cool to use this to set up integration tests for non-eth-tester nodes too!

How can it be fixed?

It would be sweet to have an easy way to snapshot state into a genesis file (which can then be reimported using #129).

I think a good API is: genesis_dict = tester.genesis_snapshot(). First step is consensus on that.

eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible.

could this strict requirement be made more generous?

pip install -r requirements.txt
...
eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible.
...

also related to the same issue as yesterday --> energywebfoundation/ew-link-origin#6 "dependencies: eth-tester 0.1.0b5 has requirement cytoolz==0.8.2, but you'll have cytoolz 0.9.0.1 which is incompatible. "

thanks

Mismatch on the acceptable input/output for contract creation transactions

  • Version: 0.1.0-beta.20
  • Python: 3.6

What was wrong?

This is most noticeable when posting a pending transaction and then mining it later:

  • You post the pending transaction that creates a contract (with no to key)
  • When retrieving the pending transaction to add to the block, the normalizer returns the transaction with to set to b''
  • The inbound normalizer/validator chokes on a to value that is not a normal address

cc @mhchia

How can it be fixed?

Since eth-tester generally follows the practice of strict requirements on both inputs and outputs, one way to solve this would be:

  1. Specify an empty to field by either:
    a. leaving it out of the transaction
    b. including a 'to' key in the transaction dict, with value: ''
  2. Normalization should update to include
    a. inbound normalization should normalize '' to b''
    a. inbound normalization should normalize a missing key to b''
    b. outbound normalization should normalize from b'' to ''
  3. When saving pending transactions to mine, do not apply outbound normalization first It's ok as is, because the transaction gets inbound normalized when applying to the block

Coverage and Gas Profiling

Most likely not the right project for this, but I couldn't think of the best project (between eth-tester, web3.py, and populus) for this feature suggestion when it comes down to implementation.


There should be a standard method to measure meaningful code-coverage metrics (Statement, Branch, and MC/DC ideally) and profile gas costs during runtime execution on EVM bytecode. Doing so on EVM bytecode can be more accurate than against source code (see here), and using a common, low-level implementation we can avoid duplication of efforts between the different langauges. With bytecode-level metrics, one can simply collect the results and figure out the bytecode to source code mapping for a specific language for reporting purposes via whatever collation method makes sense, which might be integrated into a compiler or external tool for presentation to the user.

Gas cost profiling might not make sense in this context, because IIRC they should be more or less standard for a given set of opcodes (basically everything except external calls could be analyzed via static analysis). However, I think it might actually be useful to profile at this level given that the run-time execution would figure out the costs of external calls. Collecting this cost for each transaction made since a baseline (e.g. start of testing) could report min/max/average costs for these external calls.

The best reasoning I can give for why this would be good to include in eth-tester is that it can provide this functionality at the transaction-level (basically "runtime"), which is more appropiate for testing than via static analysis methods or implementations alongside source code (like solidity-coverage).


If there is indeed tools for doing this already that are easy to use and embeddable in the testing process, please let me know. I need them for my current project.

Transaction Logs parameters naming

I am using this package in conjunction with:
web3==3.15.0
ethereum==1.6.1

When I create a transaction that has a log, the log has the following structure:
[AttributeDict({'block_hash': ... , transaction_hash: ..., log_index: ... })]

When I retrieve a transaction log from an existing HTTPProvider instance, the log has this structure:
[AttributeDict({'blockHash': ... , transactionHash: ..., logIndex: ... })]

Therefore, when I try to call from web3.utils.events.get_event_data it fails with the following error:
Traceback (most recent call last): File "/home/artemis/Documents/LiClipse_Workspace/django-ethereum-events/django_ethereum_events/decoder.py", line 39, in decode_logs decoded.append(self.decode_log(log)) File "/home/artemis/Documents/LiClipse_Workspace/django-ethereum-events/django_ethereum_events/decoder.py", line 33, in decode_log return log_topic, get_event_data(self.topics_map[log_topic]['EVENT_ABI'], log) File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/eth_utils/string.py", line 85, in inner return force_obj_to_text(fn(*args, **kwargs)) File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/web3/utils/events.py", line 201, in get_event_data 'logIndex': log_entry['logIndex'], File "/home/artemis/.virtualenvs/django_ethereum_events/lib/python3.5/site-packages/web3/utils/datastructures.py", line 25, in __getitem__ return self.__dict__[key] KeyError: 'logIndex'

Validate Transaction Function Doesnt allow for chainId

  • Version: 0.1.0b13
  • Python: 3.5
  • OS: linux

What was wrong?

The validate_transaction function does not allow for chainId as a parameter. When I ran a test using web3.py to send a transaction that included the chainId as a parameter, I received the following error.

eth_tester.exceptions.ValidationError: Only the keys 'data/from/gas/gas_price/nonce/to/value' are allowed. Got extra keys: 'chainId'

ethereum/web3.py/pull/595

How can it be fixed?

Add chainId to the inbound.py and oubound.py validation functions.

Support nonce in transactions

What was wrong?

When sending transactions from web3.py buildTransaction() with nonce specified, I got got an error from validate_transaction():

    def validate_transaction(value, txn_type):
        if txn_type not in ALLOWED_TRANSACTION_TYPES:
            raise TypeError("the `txn_type` parameter must be one of send/call/estimate")
        if not is_dict(value):
            raise ValidationError("Transaction must be a dictionary.  Got: {0}".format(type(value)))
    
        unknown_keys = tuple(sorted(set(value.keys()).difference(
            TRANSACTION_TYPE_INFO[txn_type],
        )))
        if unknown_keys:
            raise ValidationError(
                "Only the keys '{0}' are allowed.  Got extra keys: '{1}'".format(
                    "/".join(tuple(sorted(TRANSACTION_TYPE_INFO[txn_type]))),
>                   "/".join(unknown_keys),
                )
            )
E           eth_tester.exceptions.ValidationError: Only the keys 'data/from/gas/gas_price/to/value' are allowed.  Got extra keys: 'nonce'

How can it be fixed?

Add support for nonce in transaction.

This commit should be reverted when the issue is fixed.
ethereum/web3.py@07c446e

Convert documentation to ReadTheDocs

What was wrong?

Docs are currently only in the README.

How can it be fixed?

Convert documentation to be sphinx based and host on readthedocs.

PyEthereum2.1 backend tests need a different environment

  • Version: 0.1.0-beta.26
  • OS: CI

What was wrong?

See https://circleci.com/gh/ethereum/eth-tester/891?utm_campaign=vcs-integration-link&utm_medium=referral&utm_source=github-build-link

==================================== ERRORS ====================================
_____________ ERROR collecting tests/backends/test_pyethereum21.py _____________
tests/backends/test_pyethereum21.py:5: in <module>
    from eth_tester import (
eth_tester/__init__.py:4: in <module>
    from .main import (  # noqa: F401
eth_tester/main.py:29: in <module>
    from eth_tester.backends import (
eth_tester/backends/__init__.py:18: in <module>
    from .pyethereum.v20 import (
eth_tester/backends/pyethereum/v20/__init__.py:6: in <module>
    from .main import (  # noqa: F401
eth_tester/backends/pyethereum/v20/main.py:56: in <module>
    from ethereum.tools.tester import (
.tox/py36-pyethereum21/lib/python3.6/site-packages/ethereum/tools/__init__.py:1: in <module>
    from ethereum.tools import keys, new_statetest_utils, testutils, tester, _solidity
.tox/py36-pyethereum21/lib/python3.6/site-packages/ethereum/tools/keys.py:9: in <module>
    scrypt = __import__('scrypt')
.tox/py36-pyethereum21/lib/python3.6/site-packages/scrypt/__init__.py:1: in <module>
    from .scrypt import *
.tox/py36-pyethereum21/lib/python3.6/site-packages/scrypt/scrypt.py:15: in <module>
    _scrypt = cdll.LoadLibrary(imp.find_module('_scrypt')[1])
/usr/local/lib/python3.6/ctypes/__init__.py:426: in LoadLibrary
    return self._dlltype(name)
/usr/local/lib/python3.6/ctypes/__init__.py:348: in __init__
    self._handle = _dlopen(self._name, mode)
E   OSError: libcrypto.so.1.0.0: cannot open shared object file: No such file or directory

How can it be fixed?

Force an environment that has the libcrypto.so.1.0.0 library available. See some similar (but not exactly the same) environment issues in web3.py: ethereum/web3.py@8327480

Feature: Configurable transaction failure handler

Currently, when a transaction in eth-tester throws an exception (at least in pytest), it provides the entire backtrace of the failure, which is mostly showing the layers of calls back to py-evm, most of which is sort of meaningless to the user.

It would be more helpful to a user of eth-tester if there was a handler for this behavior that could given the execution trace or something so tools like @jacqueswww's vdb could handle this exception, or more generally a better trace could be produced (the EVM stack trace instead of the Python trace).

If you make this configurable, different compilers could provide language-specific behavior from giving the source code mapping, which eth-tester could use to retrace the exact line in the source program that caused the issue. This would be MUCH more helpful to debug. The default could just be an EVM opcode printer that shows the raw stack trace.

This is my thinking from sort of a high level user what they would want to see. May make more sense in the pytest-ethereum tool, but I could see this be more broadly useful and enable a general-purpose debugging API for other tools to leverage.

Transaction not mined when running multiple unit tests (PyEthereum21Backend)

  • Version: 0.1.0b26
  • Python: 3.6
  • OS: linux

Suppose the following django project structure:

  • api
    • tests.py
  • client
    • tests.py

Also suppose the following TestCase setup for both test files:

    def setUp(self):
        # Retrieve a fresh client copy
        self.client = MyClient(
            provider=self.provider, ...
        )

        super().setUp()

    def tearDown(self):
        # Revert to the clean state for every test case
        self.eth_tester.revert_to_snapshot(self.clean_state_snapshot)
        super().tearDown()

    @classmethod
    def setUpTestData(cls):
        """Only once setup"""
        cls.eth_tester = EthereumTester(backend=PyEthereum21Backend())
        cls.provider = EthereumTesterProvider(cls.eth_tester)
        cls.web3 = Web3(cls.provider)

        # deploying smart contracts etc ...

        cls.clean_state_snapshot = cls.eth_tester.take_snapshot()

Below is the output of running each TestCase on its own:

# client.tests output
Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (c1174bae) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98020 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (d4d6425a) with 1 txs and 98020 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.762s

OK
Preserving test database for alias 'default'..

and for the api tests:

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (2d599c9a) with 1 txs and 98148 gas 
.
----------------------------------------------------------------------
Ran 6 tests in 1.652s

OK
Preserving test database for alias 'default'...

Everything looks ok.
The problem appears when I try to run both test cases at once, using a single python3 manage.py test command.

What will happen is that inside my code, a call to web3.eth.waitForTransactionReceipt will always timeout or, a call to EthereumTester.get_transaction_receipt will always raise a TransactionNotFound

Below is the output of executing both test cases together, and me pressing ctrl+c to terminate when the call to web3.eth.waitForTransactionReceipt blocks ( setting the timeout to 1 second for example will just make all my unit tests fail )

Failed to import bitcoin. This is not a fatal error but does
mean that you will not be able to determine the address from
your wallet file.
Using existing test database for alias 'default'...
System check identified no issues (0 silenced).
Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
.INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
INFO:eth.block	Block pre-sealed, 98148 gas used 
INFO:eth.chain	Adding to head head=08aa138d
Saved 7 address change logs
INFO:eth.chain	Added block 6 (0fa9f473) with 1 txs and 98148 gas 
.Initializing chain from provided state
Initializing chain from provided state
INFO:eth.block	Block pre-sealed, 21000 gas used 
INFO:eth.chain	Adding to head head=c6745cf3
Saved 3 address change logs
INFO:eth.chain	Added block 1 (5f5f608a) with 1 txs and 21000 gas 
INFO:eth.block	Block pre-sealed, 470791 gas used 
INFO:eth.chain	Adding to head head=5f5f608a
Saved 2 address change logs
INFO:eth.chain	Added block 2 (3c04531f) with 1 txs and 470791 gas 
INFO:eth.block	Block pre-sealed, 1641161 gas used 
INFO:eth.chain	Adding to head head=3c04531f
Saved 2 address change logs
INFO:eth.chain	Added block 3 (94a8e4d5) with 1 txs and 1641161 gas 
INFO:eth.block	Block pre-sealed, 505467 gas used 
INFO:eth.chain	Adding to head head=94a8e4d5
Saved 2 address change logs
INFO:eth.chain	Added block 4 (92e1a399) with 1 txs and 505467 gas 
INFO:eth.block	Block pre-sealed, 283526 gas used 
INFO:eth.chain	Adding to head head=92e1a399
Saved 4 address change logs
INFO:eth.chain	Added block 5 (08aa138d) with 1 txs and 283526 gas 
^C^CTraceback (most recent call last):
  File "manage.py", line 15, in <module>
    execute_from_command_line(sys.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
    utility.execute()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 26, in run_from_argv
    super().run_from_argv(argv)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
    output = self.handle(*args, **options)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/core/management/commands/test.py", line 59, in handle
    failures = test_runner.run_tests(test_labels)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 603, in run_tests
    result = self.run_suite(suite)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/runner.py", line 569, in run_suite
    return runner.run(suite)
  File "/usr/lib/python3.6/unittest/runner.py", line 176, in run
    test(result)
  File "/usr/lib/python3.6/unittest/suite.py", line 84, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/suite.py", line 122, in run
    test(result)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/django/test/testcases.py", line 206, in __call__
    super().__call__(result)
  File "/usr/lib/python3.6/unittest/case.py", line 653, in __call__
    return self.run(*args, **kwds)
  File "/usr/lib/python3.6/unittest/case.py", line 605, in run
    testMethod()
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 110, in test_create_identity
    log = self._create_identity(owner, owner)
  File "/home/artemis/PycharmProjects/uport_relay_server/client/tests/test_client.py", line 91, in _create_identity
    tx_receipt = self.web3.eth.waitForTransactionReceipt(tx_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 199, in waitForTransactionReceipt
    return wait_for_transaction_receipt(self.web3, transaction_hash, timeout)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/utils/transactions.py", line 54, in wait_for_transaction_receipt
    txn_receipt = web3.eth.getTransactionReceipt(txn_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/eth.py", line 204, in getTransactionReceipt
    [transaction_hash],
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 103, in request_blocking
    response = self._make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/manager.py", line 86, in _make_request
    return request_func(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py", line 18, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/attrdict.py", line 18, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/normalize_errors.py", line 9, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 48, in apply_formatters
    response = make_request(method, formatted_params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/middleware.py", line 322, in middleware
    return make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/fixture.py", line 12, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/middleware/formatting.py", line 50, in apply_formatters
    response = make_request(method, params)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/main.py", line 46, in make_request
    response = delegator(self.ethereum_tester, params)
  File "cytoolz/functoolz.pyx", line 758, in cytoolz.functoolz.excepts.__call__
  File "cytoolz/functoolz.pyx", line 491, in cytoolz.functoolz.Compose.__call__
  File "cytoolz/functoolz.pyx", line 232, in cytoolz.functoolz.curry.__call__
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/web3/providers/eth_tester/defaults.py", line 36, in call_eth_tester
    return getattr(eth_tester, fn_name)(*fn_args, **fn_kwargs)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/main.py", line 336, in get_transaction_receipt
    raw_receipt = self.backend.get_transaction_receipt(raw_transaction_hash)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 376, in get_transaction_receipt
    transaction_hash,
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/eth_tester/backends/pyethereum/v20/main.py", line 155, in _get_transaction_by_hash
    if transaction.hash == transaction_hash:
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/transactions.py", line 141, in hash
    return utils.sha3(rlp.encode(self))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 192, in sha3
    return sha3_256(to_string(seed))
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/ethereum/utils.py", line 4, in sha3_256
    def sha3_256(x): return keccak.new(digest_bits=256, data=x).digest()
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 173, in new
    return Keccak_Hash(data, digest_bytes, update_after_digest)
  File "/home/artemis/.virtualenvs/uport_relay_server/lib/python3.6/site-packages/Crypto/Hash/keccak.py", line 70, in __init__
    state = VoidPointer()
  File "/usr/lib/python3.6/unittest/signals.py", line 36, in __call__
    self.default_handler(signum, frame)
KeyboardInterrupt

Tools for testing chain reorgs

What was wrong?

It would be nice to be able to test chain-reorg in a simple manner. I'm not sure where in the stack the fits best but something like

EthereumTester.mine_reorg(reorg_depth=1)

might work.

Main use case would be testing scenarios where state is updated from block contents and has to be updated when the chain changes.

py-evm backend ignores fork setup

What was wrong?

As of alpha 26, there is no clean way to define the VM configuration at runtime. MainnetTesterChain.configure_forks is broken: it sets the configuration on the vm instance, and then get_vm_class_for_block_number ignores vm_configuration on the instance, using the class's vm_configuration instead.

How can it be fixed?

No one noticed that fork choice stuff isn't working. Maybe just neuter the API to officially make it do nothing, and we can re-add the functionality when it's requested.

Also, probably remove MainnetTesterChain.configure_forks from py-evm since it's described as "footgun" and isn't working anyway.

eth-tester[pyethereum16] filter 'toBlock' is exclusive

  • Version: eth-tester==0.1.0b21
  • Python: 3.6
  • OS: osx

What was wrong?

I'm not sure where to report this best, but as it happens on a eth-tester based chain I think this is a good place.

The issue is that creating a filter with the toBlockparameter set to a block number behaves differently when eth-tester with the pyethereum1.6 backend than with geth.

Geth's toBlock is inclusive while the eth-tester filter excludes it. Even though this is a backend related problem this should probably be handled by eth-tester or the backend plugins and tested.

How can it be fixed?

Handle filter behaviour and test that toBlock is inclusive.

MoackBackend returned hash value can't found valid trans

  • Version: 0.1.0b32
  • Python: 3.5
  • OS: win

What was wrong?

I got below error:

Traceback (most recent call last):
  File "D:\eth-tester\eth_tester\backends\mock\main.py", line 244, in get_transaction_receipt
    receipt = self.receipts[transaction_hash]
KeyError: b'\xf3\xbc\xde#\xb2U\x05H\xe0\x98.\xae\xb5}\xb3\x80!\xd4\xca\x0f6\xd8\xc9\xb1\x05\xa3<&\x9eR\x91\x9b'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:/eth-tester/sample.py", line 10, in <module>
    print(t.get_transaction_receipt('0xf3bcde23b2550548e0982eaeb57db38021d4ca0f36d8c9b105a33c269e52919b'))
  File "D:\eth-tester\eth_tester\main.py", line 307, in get_transaction_receipt
    raw_receipt = self.backend.get_transaction_receipt(raw_transaction_hash)
  File "D:\eth-tester\eth_tester\backends\mock\main.py", line 247, in get_transaction_receipt
    "No transaction found for hash: {0}".format(transaction_hash)
eth_tester.exceptions.TransactionNotFound: No transaction found for hash: b'\xf3\xbc\xde#\xb2U\x05H\xe0\x98.\xae\xb5}\xb3\x80!\xd4\xca\x0f6\xd8\xc9\xb1\x05\xa3<&\x9eR\x91\x9b'

Process finished with exit code 1

My code is below:

from eth_tester import EthereumTester, MockBackend


t = EthereumTester(MockBackend())
print(t.get_accounts())
print(t.get_balance('0xaBbACadABa000000000000000000000000000000'))
print(t.send_transaction({'from': '0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1', \
                          'to': '0x7d577a597B2742b498Cb5Cf0C26cDCD726d39E6e', 'gas': 21000, 'value': 1}))
print(t.get_transaction_receipt('0xf3bcde23b2550548e0982eaeb57db38021d4ca0f36d8c9b105a33c269e52919b'))

How can it be fixed?

I wonder if this issue only exist on MockBackend or on PyEVM both?

Calling Filter.remove(*values) with no values produces unhandled error

  • Version: 0.1.0b27
  • Python: 3.6.5
  • OS: linux

What was wrong?

I am using the snapshot functionality to catch and revert a TransactionFailed error when automining transactions. I often (70% of the time in my test suite) run into an issue when reverting to a snapshot where it catches the following unhandled exception in this function call:

def remove(self, *values):
try:
values_to_remove = set(values)
except TypeError:
# log filters are dicts which are not hashable
values_to_remove = values
queued_values = self.get_changes()
self.values = [
value
for value
in self.get_all()
if value not in values_to_remove
]

    self._t.revert_to_snapshot(self._snapshot_id)
eth_tester/main.py:543: in revert_to_snapshot
    self._revert_log_filter(log_filter)
eth_tester/main.py:590: in _revert_log_filter
    filter.remove(*values_to_remove)
eth_tester/utils/filters.py:62: in remove
    in self.get_all()

---------------------------------------------------------------------------------------------

.0 = <tuple_iterator object at 0x7f2585dfdef0>

    value
    for value
    in self.get_all()
>   if value not in values_to_remove
            ]
E           TypeError: unhashable type: 'dict'

eth_tester/utils/filters.py:63: TypeError

When this error occurs, the following variables of interest are set:

values = ()  # Empty tuple

# Tuple of 1 dict
self.get_all() = ({'type': 'mined', 'log_index': 0, 'transaction_index': 0, 'transaction_hash': b'_\x0f;\xd1n/@pq\xcaL\xea\xd1\x11\xe5\x86\x03\xc3\xe9.\xd4\x88\x8b9\xfaHD\xc3\xa3;V\x1d', 'block_hash': b'\xcf\x9b\xec\xcaD[\x7fm\xc0KJ\x1c{\x80.Z\t\xa8\xb7fV\xe5\xb9X\x05\xe0N\x85E\xb0\xc0.', 'block_number': 2, 'address': b'\xf2\xe2F\xbbv\xdf\x87l\xef\x8b8\xae\x84\x13\x0fOU\xde9[', 'data': b'', 'topics': (b'\x8b\xe0\x07\x9cS\x16Y\x14\x13D\xcd\x1f\xd0\xa4\xf2\x84\x19I\x7f\x97"\xa3\xda\xaf\xe3\xb4\x18okdW\xe0', b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+Z\xd5\xc4y\\\x02e\x14\xf81|z!^!\x8d\xcc\xd6\xcf', b'\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00+Z\xd5\xc4y\\\x02e\x14\xf81|z!^!\x8d\xcc\xd6\xcf')},)

values_to_remove = set()  # Empty set

How can it be fixed?

I haven't been able to replicate this issue as it is inconsistent for me during runtime in my testing tool, but I think handling when values is unset is fairly obvious, since this basic Python also raises the same error:

[v for v in ({},) if v not in set()]

TypeError: unhashable type: 'dict'

I can submit a quick fix for this, but I need a little more instruction on how to handle this scenario

I am thinking of suggesting the following:

    queued_values = self.get_changes()
    if values_to_remove:
        self.values = [
            value
            for value
            in self.get_all()
            if value not in values_to_remove
        ]
        for value in queued_values:
            if value in values_to_remove:
                continue
           self.queue.put_nowait(value)
    else:
        self.values = self.get_all()
        for value in queued_values:
           self.queue.put_nowait(value)

Something smarter with filtering probably would work too.

Accept empty array of addresses in filter, match any address

What was wrong?

The spec is unclear about what to do with a filter that has an empty list of addresses. At least one reference implementation (geth) chooses to interpret an empty list of addresses as matching all addresses: https://github.com/ethereum/go-ethereum/blob/461291882edce0ac4a28f64c4e8725b7f57cbeae/eth/filters/filter.go#L254-L256

How can it be fixed?

Given that the spec is unclear, let's just match geth's implementation. (Let's re-open discussion if parity does something different)

eth-tester should accept an empty array of addresses in a filter, and treat it the same as if no addresses key was supplied.

Cleanup address and account utils

What is wrong?

  • There is a duplicate util in both eth_tester.utils.account and eth_tester.utils.address for generating contract addresses.
  • The private_key_to_address function should be removed in favor of using eth-keys API

Better API for initializing with genesis parameters

What was wrong?

As of #123 there will be a way to specify genesis parameters, but it will take some detailed documentation to understand, and the variety of options will probably confuse beginners. (Thanks to @voith for offering to write the docs)

Let's reduce the ways to initialize the backend with custom genesis parameters, and do it in a way that leans more on the global standards (like the genesis file, which may only be a "de facto" standard, but still better than a custom web3.py thing).

How can it be fixed?

Let's can coalesce around this as the preferred single mechanism for initializing a custom genesis state:

backend = PyEVMBackend.from_genesis(dict(gas_limit=etc))

# also accepts the equivalent json-encoded string:
backend = PyEVMBackend.from_genesis('{"gas_limit": etc}')

We should further explore what use cases people have for setting up custom genesis, but I think we can handle them all cleanly by creating tools to easily generate a genesis dict.

The following APIs are shooting from the hip, I'm not saying that any specific one is a good idea. Just some examples...

from eth_utils import make_genesis_accounts

genesis_accounts = make_genesis_accounts(num_accounts=3, init_state=dict(balance=10**18))

PyEVMBackend.from_genesis(dict(gas_limit=7_000_000, accounts=genesis_accounts))

or

# some silly keys, where int(key) in range(10)
keys = eth_utils.get_test_keys(10)

genesis = eth_utils.make_funded_genesis(dict(gas_limit=7_000_000), keys=keys)

tester_backend = PyEVMBackend.from_genesis(genesis)

for key in keys:
  tester_backend.add_account(keys)

How cool is it that the same tools could be used to generate a genesis file for any other node?! Speaking of which, maybe these generation tools already exist somewhere else?

Vyper dependency causing testing errors

  • Version: 0.1.0b24/0.1.0b26
  • Python: 3.6.5
  • OS: osx

What was wrong?

Testing errors when bumping version of vyper

Reproduce:

  • pull down casper master
  • in requirements.txt, bump vyper version to most recent commit โ€” 8113f79f7bce362e7a0f803b32736cb460c78640
  • install via pip install -r requirements.txt
  • run pytest tests

The above will produce the following error:

 TypeError: 'ChainDB' object does not support item assignment

If the version of eth-tester is bumped to 0.1.0b26 in addition to the vyper commit bump, running pytest tests will produce:

evm.exceptions.ValidationError: mix hash mismatch; 0x351006e4ee3e28295bf07ff25dd9fd91d23751e0036fdfc39a51635e0e2edd69 != 0x0000000000000000000000000000000000000000000000000000000000000000

How can it be fixed?

Not sure yet.

README.md contains errors, need check and fix

I have git clone latest code , and found issues for README.md
My dev software is Windows+Python3.5

Issue1

Development
pip install -e . -r requirements-dev.txt
but i can't find requirements-dev.txt, and got below error:
Could not open requirements file: [Errno 2] No such file or directory: 'requirements-dev.txt'

Issue2

below are part of Quick Start in README.md

t.send_transaction({'from': '0x82A978B3f5962A5b0957d9ee9eEf472EE55B42F1', 'to': '0x7d577a597B2742b498Cb5Cf0C26cDCD726d39E6e', 'gas': 21000, 'value': 1})
'0x140c1da1370a908e4c0f7c6e33bb97182011707c6a9aff954bef1084c8a48b25'
t.get_transaction_by_hash(0x140c1da1370a908e4c0f7c6e33bb97182011707c6a9aff954bef1084c8a48b25')
t.t.get_transaction_by_hash(0x...) missing one '

Issue3

t.get_block_by_numbers(1)
this func has been replaced by t.get_block_by_number(1)

I hope anyone can check README.md, and execute all process once, to see if exists any other erros.

Runtime error

  • Version: 0.1.0b23
  • Python: 3.6.4
  • OS: MacOS Sierra 10.12.16

What was wrong?

When using eth_tester as part of our test suite, I get some runtime errors as exemplified by the following stack trace:

    price,
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/contract.py:1311: in transact_with_contract_function
    txn_hash = web3.eth.sendTransaction(transact_transaction)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/eth.py:244: in sendTransaction
    [transaction],
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/manager.py:103: in request_blocking
    response = self._make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/manager.py:86: in _make_request
    return request_func(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/gas_price_strategy.py:18: in middleware
    return make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/attrdict.py:18: in middleware
    response = make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/normalize_errors.py:9: in middleware
    result = make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/validation.py:43: in middleware
    return make_request(method, post_validated_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/middleware.py:320: in middleware
    return make_request(method, [filled_transaction] + params[1:])
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/fixture.py:12: in middleware
    return make_request(method, params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/middleware/formatting.py:21: in middleware
    response = make_request(method, formatted_params)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/main.py:46: in make_request
    response = delegator(self.ethereum_tester, params)
cytoolz/functoolz.pyx:232: in cytoolz.functoolz.curry.__call__
    ???
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/web3/providers/eth_tester/defaults.py:36: in call_eth_tester
    return getattr(eth_tester, fn_name)(*fn_args, **fn_kwargs)
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:78: in func_wrapper
    self.mine_block()
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:394: in mine_block
    block_hash = self.mine_blocks(1, coinbase=coinbase)[0]
../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:389: in mine_blocks
    self._process_block_logs(block)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <eth_tester.main.EthereumTester object at 0x113774748>, block = {'difficulty': 131200, 'extra_data': '0x0000000000000000000000000000000000000000000000000000000000000000', 'gas_limit': 998051069, 'gas_used': 266681, ...}

    def _process_block_logs(self, block):
>       for _, filter in self._log_filters.items():
E       RuntimeError: dictionary changed size during iteration

../../../../.pyenv/versions/3.6.4/lib/python3.6/site-packages/eth_tester/main.py:401: RuntimeError

Ideas?

dependency cycle with py-evm backend

  • Version: 0.1.0b23
  • Python: 3.6
  • OS: osx

What was wrong?

I specify eth-tester[py-evm]>=0.1.0-beta.23,<1.0 in my dependencies. pip install -r requirements.txt works fine with that.

However, when trying to run python setup.py develop on my package I get the following error:

error: eth-tester 0.1.0b23 is installed but eth-tester==0.1.0b21 is required by {'py-evm'}

eth-tester requires py-evm 0.2.0a14, but this one in return requires eth-tester 0.1.0b21: https://github.com/ethereum/py-evm/blob/cc877851d4e110eabfba1bfe544bfe187b51ebea/setup.py#L29

How can it be fixed?

Fix the dependency cycle.

Write a web3 backend

Write a backend which uses web3. This sets up the ability to test against real nodes (with limited functionality since we can't time travel a real node).

Remove pyethereum backends

What was wrong?

The pyethereum codebase is unmaintained.

How can it be fixed?

Drop both of the pyethereum based backends.

Ability to generate blocks with uncles

What was wrong?

Generating a block with an uncle isn't something that can really be done in any reasonable way.

How can it be fixed?

Add an API for generating uncles.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.