This repository was merged into https://github.com/neo-project/neo, newer (post-3.6.2) VM versions can be obtained from it.
This is the virtual machine of NEO: NeoVM.
For detailed documentation, see docs.neo.org and the NEO whitepaper.
NEO Virtual Machine
License: MIT License
This repository was merged into https://github.com/neo-project/neo, newer (post-3.6.2) VM versions can be obtained from it.
This is the virtual machine of NEO: NeoVM.
For detailed documentation, see docs.neo.org and the NEO whitepaper.
We need to have some sort of interface for ExecutionEngine, in order to better test features inside Neo without needing to have a full system set-up. If we have that interface, we can use mock and test only the desired features when necessary (example, verify if a given Interop actually pushes on stack the expected value).
@erikzhang This instruction shouldn't be a Peek? if you pop the key, value and array, where goes the result?
neo-vm/src/neo-vm/ExecutionEngine.cs
Line 753 in b1338d4
Is necessary to clone first the array for call this opCode?
Like CAT, SUBSTR and LEFT, it would be nice to have an opcode for reversing a byte array. Useful for script hashes.
As discussed with community, it would be useful to have standard test format to guarantee cross-compatible integration tests between projects.
I believe current JSON tests are very nice, and I have started to build a validation JSON schema for it. By doing that, some small ideas emerged to fix and clarify details.
Right now, the following draft can already be validated:
{
"category": "Arrays",
"name": "ARRAYSIZE",
"tests": [
{
"name": "Wrong type SYSCALL[System.ExecutionEngine.GetScriptContainer]+ARRAYSIZE",
"script": "0x682953797374656D2E457865637574696F6E456E67696E652E476574536372697074436F6E7461696E6572C0",
"message": "0x00"
}
]
}
The proposed draft schema is:
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://neo.org/neovm.json",
"type": "object",
"title": "NeoVM Tests",
"required": [
"category",
"name",
"tests"
],
"properties": {
"category": {
"$id": "#/properties/category",
"type": "string",
"title": "test category (rename to 'suite' name?)",
"default": "",
"examples": [
"Arrays"
],
"pattern": "^(.*)$"
},
"name": {
"$id": "#/properties/name",
"type": "string",
"title": "Name for test suite",
"default": "",
"examples": [
"ARRAYSIZE"
],
"pattern": "^(.*)$"
},
"tests": {
"$id": "#/properties/tests",
"type": "array",
"title": "Tests for specific suite",
"items": {
"$id": "#/properties/tests/items",
"type": "object",
"title": "Each test case",
"required": [
"name",
"script",
"message"
],
"properties": {
"name": {
"$id": "#/properties/tests/items/name",
"type": "string",
"title": "specific test name (rename to description?)",
"default": "",
"examples": [
"Wrong type SYSCALL[System.ExecutionEngine.GetScriptContainer]+ARRAYSIZE"
],
"pattern": "^(.*)$"
},
"script": {
"$id": "#/properties/tests/items/script",
"type": "string",
"title": "specific test script",
"default": "",
"examples": [
"0x682953797374656D2E457865637574696F6E456E67696E652E476574536372697074436F6E7461696E6572C0"
],
"pattern": "^(0x)?([0-9A-F][0-9A-F])*$"
},
"message": {
"$id": "#/properties/tests/items/message",
"type": "string",
"title": "specific test message",
"default": "",
"examples": [
"0x00"
],
"pattern": "^(0x)?([0-9A-F][0-9A-F])*$"
}
}
}
}
}
}
Material on JSON schema may be found here: https://json-schema.org/understanding-json-schema
Validation can be experimented here: https://www.jsonschemavalidator.net
Right now, my only suggestions are to rename "category" to "suite" (or "testsuite"), and "name" to "description" (inside tests). I also suggest the inclusion of a field "version", to evolve this standard in the future. The part of "steps" is still unfinished, it will take a little work, but I think it can be finished very soon.
A StackItem of type Boolean with a false
state returns 0 when calling GetByteLength()
on it. This seems counter intuitive as false
is still a value and takes up space in memory. Should the Boolean StackItem type not always return a length of 1?
My rationale is; I expect GetByteArray()
to return 1 byte set to 0
for false
and set to 1
for true
, this would then result in GetByteLength()
to always be 1
. Neither of these assumptions is true (meaning; GetByteArray()
on a Boolean with a false
state, returns an empty bytearray). If this is intentional, why is it like that?
We propose an operation for Elliptic curve point scalar multiplication and (possibly) addition. The idea is to allow onchain implementation of zero knowledge and other cryptography techniques, allowing users to switch prefered curves.
This is related to MODPOW issue, and also could be interop, if community prefers.
The intented syntax is to receive curve parameters on a input array on stack, followed by point (array of two items) and scalar (another bytearray).output will be a point (array of two elements).
I'm here to propose an NVM Extension Proposals (like a NEP in fact), that will allow us to introduce new features, in the future NVM3, in a very controlled way.
I think that after NeoVM 3 is released, not much changes will happen on its opcodes in a near future, it should be very stable. However, there will always appear new ideas and extensions that would better fit here on NVM, than on the ApplicationEngine.
I propose a single new opcode EXTENSIONS
, that receives two bytes as parameter, allowing up to 65k possible extensions, grouped by category (first byte), and another one for specifics (otherwise 0x00).
This will work like SYSCALLS, but in a very lightweight manner, and constrained to NeoVM project itself (not mentioning specific Neo Blockchain capabilities). In order to create an EXTENSION, a new NEP must be created declaring it, and after that, it could be implemented on NVM project.
Versioning is not proposed here, meaning that it is the responsibility of upper layer and dependent projects to care how they manage their NVM usage. The only thing NVM will provide is disable specific extensions, in a map format.
I think that NVM 3 fulfilled it's intention of being a lightweight virtual machine, however I think it should embed capabilities related to a verifiable execution engine, meaning that it should be able to embed some elaborate verifiable data structures, hashing and cryptography.
This should allow NVM to become a standard lightweight vm for blockchain and verifiable computation, as there's no such existing vm nowadays.
Therefore, my first proposal would be to embed the following extensions:
0x0101
-> SHA256
0x0201
-> secp256r1 verification
We could start a NEP with that.
I have a main method like this:
public static object Main(string operation, params byte[][] args)
and would like to prime args
in a unit test. I've tried something like this:
byte[] program = // script is loaded here
var engine = new ExecutionEngine(null, Crypto.Default);
engine.LoadScript(program);
using (ScriptBuilder sb = new ScriptBuilder()) {
sb.EmitPush(new byte[] { 1, 2, 3, 4, 5 }); // args[0]
sb.EmitPush(1);
sb.EmitPush(OpCode.PACK);
sb.EmitPush("operation"); // operation
engine.LoadScript(sb.ToArray());
}
engine.Execute();
This doesn't work but seems like it should.
Many applications (so many!) could benefit of a jump-free execution mode, where execution is bound to the number of opcodes, at most. Could we provide such a way now? It could be a flag passed to the execution engine, e.g. nojumps. This would also require disabling recursive calls and perhaps all calls.
Dear all, I've been thinking on this for a while, and I think now that a single new opcode can help solving many different issues.
I propose the creation of a new opcode, CONVERT <type>
, that receives a byte <type>
indicating the conversion type and converts element that is on top stack (putting the converted element on stack). I initially propose the following types:
0x00: ByteArray
0x01: Array
0x02: Integer
0x03: Struct
0x04: Map
0x05: Boolean
...
0x80: Clone
Applications:
PUSH_bytearray
CONVERT 0x01 # convert to array
DUP
REVERSE
CONVERT 0x00 # convert to bytearray
We could perhaps transform array into structs and solve this: #51
CONVERT 0x03 # convert stack array to struct
We could transform bytearrays to integers and guarantee that they would follow BigInteger standard (by summing zero for example).
CONVERT 0x02 # convert to biginteger
We could convert maps and arrays, so repeated opcodes could be eliminated (including bytearray/array sizes, appends, ...)
CONVERT 0x01 # convert to array and get length
ARRAYSIZE
We could do an efficient concatenation of array elements by transforming array to bytearray
PUSH ...
PUSH ...
CAT
PUSH ...
CAT
####
PUSH ...
NEWARRAY
CONVERT 0x00 # convert to bytearray
We could perhaps add a CLONE type in conversion(0x80 + 0x03
), so we could choose to convert into a new one or an actual clone (and help solving this #63)
CONVERT 0x83 # convert to cloned struct
We could avoid this PACK/UNPACK workaround: #60
CONVERT 0x01 # convert to array and assign value
... assign ...
CONVERT 0x00 # convert back to bytearray
We can use it to enforce alternative types such as Boolean and Byte (even if stored as bytearray).
PUSH5
CONVERT 0x05 # to boolean (true if not zero)
Perhaps this feature could antecipate issues and bring many future possibilities on Neo 3.0 (and removing many unused opcodes), just by having a direct conversion tool for StackItems.
I think that is not necessary allow execute ' RET' command while only push is allowed
neo-vm/src/neo-vm/ExecutionEngine.cs
Line 56 in b1338d4
What do you think @erikzhang ?
The NUMEQUAL instruction in the new 2.4.1 version of NEOVM adds a check for size, which causes problems with some contracts that have been deployed on the MainNet.
For example, the NNC contract (0xfc732edee1efdf968c23c20a9628eaa5a6ccb934). Because the NNC contract had an upgrade operation, in which there was a comparison of new and old contract scripts. this comparison was added to indicate that the upgraded contract is a different contract. Because the content of the comparison exceeds the size limit, it causes the 2.10.1 version of the node to return fault when it executes the contract. The upgraded NNC contract can not be found in the 2.10.1 version of the node that re-synchronized data.
I want to move the limits (stack,BigInteger..) and Gas costs from ApplicationEngine to this project, NeoVM.
Benficts:
If someone want to use the NeoVM, with other limits, they could configure before use it according to his necessities.
Why is this not in ByteArray?
neo-vm/src/neo-vm/Types/Array.cs
Line 10 in 8c87712
The VM is entering a FAULT state when a SETITEM is used on my byte array. See below:
case OpCode.SETITEM:
{
StackItem newItem = EvaluationStack.Pop();
if (newItem.IsStruct)
{
newItem = (newItem as Types.Struct).Clone();
}
int index = (int)EvaluationStack.Pop().GetBigInteger();
StackItem arrItem = EvaluationStack.Pop();
if (!arrItem.IsArray) // <------ the isArray check
{
State |= VMState.FAULT; // <-- :(
return;
}
The ScriptBuilder class uses System.BitConverter to convert integer types to bytes. The output of the BitConverter.GetBytes method will be different depending on the CPU architecture (little endian vs big endian).
else if (data.Length < 0x10000)
{
Emit(OpCode.PUSHDATA2);
ms.Write(BitConverter.GetBytes((ushort)data.Length), 0, 2);
ms.Write(data, 0, data.Length);
}
else// if (data.Length < 0x100000000L)
{
Emit(OpCode.PUSHDATA4);
ms.Write(BitConverter.GetBytes((uint)data.Length), 0, 4);
ms.Write(data, 0, data.Length);
}
Erik, I think it's time to come back to this issue: #54 (comment)
Unified entry point for all verification methods. Together with this, we can abolish CHECKSIG and MULTISIG, for Neo 3.0, and implement them as Native contracts using VERIFY.
A more uniform view of the number zero can be achieved by always represent it in its most compressed format. 0x0000 is zero, 0x00 is also zero, but 0x'' (empty array) is also zero. So we should adopt empty array as our new standard. Currently, the standard (by C# BigInteger) is "0x00" I think (although 0x"" is also recognized as zero...)
This helps in many things:
If someone wants the hexstring zero, it can do PUSHBYTES1 00
, as always.
The latest VM does not seem to be working properly with certain contracts. I traced this to the CALL_I instruction.
I believe that since the recent introduction of the Instruction class, the CALL_I jump target's offset is being calculated from the instruction pointer value immediately after the CALL_I is read.
But as I read it, in the previous implementation, the CALL_I jump target's offset would be calculated from the instruction pointer value after the rvcount/pcount operand bytes are read, i.e. two bytes further into the script, which is what the existing compiler expects.
So as a result, in the new VM, all CALL_I jumps are landing two bytes short of where they should be, which would make most contracts compiled using NEP8 malfunction on most operations.
It seems to me that the description of the following opcodes does not match the code:
/// Reads a 2-byte value n and a jump is performed to relative position n-3.
/// </summary>
JMP = 0x62,
/// <summary>
/// A boolean value b is taken from main stack and reads a 2-byte value n, if b is True then a jump is performed to relative position n-3.
/// </summary>
JMPIF = 0x63,
/// <summary>
/// A boolean value b is taken from main stack and reads a 2-byte value n, if b is False then a jump is performed to relative position n-3.
/// </summary>
JMPIFNOT = 0x64,
Could you please clarify?
same remark for CALL:
/// Current context is copied to the invocation stack. Reads a 2-byte value n and a jump is performed to relative position n-3.
/// </summary>
CALL = 0x65,
we don't see any relative jump of n-3:
case OpCode.CALL:
{
if (!CheckMaxInvocationStack()) return false;
ExecutionContext context_call = LoadScript(context.Script);
context_call.InstructionPointer = context.InstructionPointer + instruction.TokenI16;
if (context_call.InstructionPointer < 0 || context_call.InstructionPointer > context_call.Script.Length) return false;
context.EvaluationStack.CopyTo(context_call.EvaluationStack);
context.EvaluationStack.Clear();
break;
}
we should add this functions to opcode.
//Syscall("System.ExecutionEngine.GetEntryScriptHash")
ENTRYSCRIPT = 0xD0,
//Syscall("System.ExecutionEngine.GetExecutingScriptHash")]
CALLINGSCRIPT = 0xD1,
//[Syscall("System.ExecutionEngine.GetCallingScriptHash")]
EXECUTINGSCRIPT = 0xD2,
this is base function,like implement static variable.
static variable is shared by same script,so we need to compare executingscript
we can store static vars like this:
map<scripthash, static vars>
so when we use a static var,opcode maybe like:
DUPFROMSTATICSTACK,//get map
EXECUTINGSCRIPT, //get key :scripthash
PICKITEM //get vars=map[key]
PUSH var id //get key:var id
PICKITEM //get var=vars[key]
I think neocompiler use syscall to get executingscript for static variable is a bad idea.
syscall is more slower than opcode.
and this will make neocompiler to write a fix syscall.
Like PACK
but for structs.
It will be immensely powerful if it supports even this most basic GPGPU op codes.
Can we start adding release tags for this repo as well? It's becoming hard to know which commits work with which neo-cli
version. Like now I want to crosscheck RPC output against neo-cli
2.10.0.0
after all the VM updates but that still returns multiple engine states as if it still were flags.
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"script": "00c1046e616d6567c2563cdd3312230722820b1681d30fe5f6cffbb9",
"state": "HALT, BREAK",
"gas_consumed": "0.128",
"stack": [
{
"type": "ByteArray",
"value": "4e45582054656d706c617465205632"
}
]
}
}
The instructions DUPFROMALTSTACK
and DUP
are currently returning a reference of a StackItem
. This is not how stack machines should work at their core, hence leaving some security vectors open as we are directly modifying a StackItem
on another stack. Which breaks all rules of stack isolation
.
We can see in the instruction APPEND
that the VM is not pushing the array back to the ALT stack. It is modifying the memory on the ALT stack directly. reference append
Both DUPFROMALTSTACK
and DUP
should return a clone by popping the item of the ALT stack. Instead of creating a referenced copy of it.
Psuedo code
Currently
dup => modify reference directly
Proposel
pop => modify clone => push back to ALT stack
If we take the APPEND
instruction for example, after appending the item to the collection we need to push the collection back to the ALT stack. there are a couple other instructions that will need some changes to make this work. However, those changes are very minor for a major improvement.
This issue occurs also in other VM implementations (py, go, ..)
neo-vm/src/neo-vm/InteropService.cs
Line 43 in 79c3aad
I think that if you call this SYSCALL
in the first script, you put a null value into the stack, maybe this is the unique point when you can add a null value to the stack, maybe we can fix it? if don't other VM like c++ VM should create a new StackItem (NullStackItem
) because could produce different behaviours
At now InteropInterface
just accept IInteropInterface
objects, but this cause a problem if you want to change the VM, because you can't share the same InteropService
. This interface is defined inside the VM package, so we can't reuse the same InteropService
for different implementations of the VM.
If we change this to a native object, or just accept any object, all should works for both scenarios
This would allow ArrayStackItem
s, MapStackItem
s, StructStackItem
s and InteropInterface
s to be used as keys in MapStackItem
s, which would be very convenient for modeling smart contract language maps. E.g. in C# or JavaScript, MapStackItem
would map 1:1 to builtin dictionaries and maps respectively.
I think that codecov is computing wrong the coverage on ExecutionEngine
class.
Opcodes like CHECKSIG
or CHECKMULTISIG
are not tested, you can verify that Crypto
methods are without unit tests https://codecov.io/gh/neo-project/neo-vm/src/master/tests/neo-vm.Tests/Types/Crypto.cs#L41 but in the switch https://codecov.io/gh/neo-project/neo-vm/src/master/src/neo-vm/ExecutionEngine.cs#L860 it appears as tested
Also in the readme, the badge appears without percentage
[EDIT] The initial ideia was to create a new opcode called CONVERT
. Then, after discussions with @shargon , I changed to a new direction, without creating any new opcode, just updating PACK and UNPACK.
=====
I think three different problems can be solved by having an opcode capable of converting byte[] to Array (of byte) and back.
The idea of this opcode CONVERT is to allow a convertion between a reference type Array to a static type bytearray, and to allow byte[] to have easy use of operations that are available for arrays. All these operations could be done in high level, however, at a big cost of opcodes, and higher code complexity. Using this opcode, REVERSE could be easily done:
PUSH10 ababababab...
CONVERT # converts byte[] to Array (of byte) on stack
DUP # duplicates Array on stack
REVERSE # reverse and drop array
CONVERT # convert Array back to byte[] on stack
SETITEM will work in the same way.
Opcode SIZE could be easily replaced by ARRAYSIZE (as proposed by Erik on #57).
PUSH10 ababababab...
CONVERT # converts byte[] to Array (of byte) on stack
DUP # duplicates Array on stack
ARRAYSIZE # removes Array and puts its size on stack
/// <summary>
/// A bytearray or array (of byte) is taken from the stack. If bytearray is taken, an array with the same bytes is put on stack. If Array is taken, a bytearray is created with the exact size, having each element transferred from one to another.
/// </summary>
CONVERT = 0xCE,
We use structs heavily in NEO-ONE TypeScript smart contracts and we depend on the fact that structs are cloned from PICKITEM
. So, we'd like to implement #49 again, however this time with a simple logic switch such that old contracts use the old buggy behavior and new contracts use the corrected behavior.
Ideally, we could just turn this feature on using the height the contract was created at, but we don't store this information anywhere. So we'll need an alternate way to determine if a given contract should use the new behavior or old. I suggest that we add a new contract property similar to payable, dynamic invoke, etc. which when present means that the contract uses the new behavior.
This is blocking for NEO-ONE TypeScript smart contracts.
When syncing from the MainNet chain file (as obtained from http://sync.ngd.network/) on multiple occasions, I randomly get exceptions to occur when deducting NEO balances (see neo-project/neo#391).
I thought it was gone after moving to the latest VM since I was able to sync once without an issue after moving to latest commit 94a91ae6 . I was originally on commit ab74c361. However, upon syncing the chain again, it happened again.
I added a richer log to see a bit more information about what was happeining. Here is an example of what I got on two separate occasions syncing the chain:
2018-09-25 19:14:41.118 F Neo.Ledger.Blockchain: Account (ANcEb4Xw8ouJnk5e5Q1z6o92Z67pNrydQ2 = 0x1e9ca763049717f04952a1c0118678098a33f64a) doesn't have a balance for asset 0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b 1 Block: 1858860
System.Collections.Generic.KeyNotFoundException: The given key '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b' was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at Neo.Ledger.Blockchain.Persist(Block block) in [redacted]/neo/Ledger/Blockchain.cs:line 532
...
...
2018-09-26 02:41:58.910 F Neo.Ledger.Blockchain: Account (AWnA7E3tNpc3vtYRWdxkHYiRBBu2mSsAYR = 0xf23af39524ce9e711eaac979a015e384a7a097a4) doesn't have a balance for asset 0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b 1 Block: 2582510
System.Collections.Generic.KeyNotFoundException: The given key '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b' was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at Neo.Ledger.Blockchain.Persist(Block block) in /home/solinsky/repos/aph-server/neo/Ledger/Blockchain.cs:line 532
[ERROR][9/26/18 2:41:58 AM][Thread 0010][akka://NeoSystem/user/$a] The given key '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b' was not present in the dictionary.
Cause: System.Collections.Generic.KeyNotFoundException: The given key '0xc56f33fc6ecfcd0c225c4ab356fee59390af8560be0e930faebe74a6daff7c9b' was not present in the dictionary.
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at Neo.Ledger.Blockchain.Persist(Block block) in [redacted]/neo/Ledger/Blockchain.cs:line 532
at Neo.Ledger.Blockchain.OnImport(IEnumerable`1 blocks) in [redacted]/neo/Ledger/Blockchain.cs:line 250
at Neo.Ledger.Blockchain.OnReceive(Object message) in [redacted]/neo/Ledger/Blockchain.cs:line 413
My best theory so far is that something happening during execution of a script was non-deterministic leaving the balance of an account in a state different from what it was at the time the consensus node forged the block using a previous version of the VM. This is a theory that needs to be confirmed. It has happened on different hardware now, so I don't think it is faulty hardware, and I'm using the same version of LevelDB I have been using for a while before having encountered this.
This is running Neo 2.9.0, maybe I should try against 2.8.0 to see if it ever happens with latest VM to see if my assumption that it is VM related is correct.
Anyway, the only difference to my PersistBlock function that threw the above exceptions is the exception handling. I added comments to the PersistBlock function also to aid in thinking through what might be happening. I'll include the top part of the function that I modified to give more info here in case someone else has more time to look into it.
This code starts on line 439 of Blockchain.cs:
private void Persist(Block block)
{
using (Snapshot snapshot = GetSnapshot())
{
snapshot.PersistingBlock = block;
snapshot.Blocks.Add(block.Hash, new BlockState
{
SystemFeeAmount = snapshot.GetSysFeeAmount(block.PrevHash) + (long)block.Transactions.Sum(p => p.SystemFee),
TrimmedBlock = block.Trim()
});
foreach (Transaction tx in block.Transactions)
{
snapshot.Transactions.Add(tx.Hash, new TransactionState
{
BlockIndex = block.Index,
Transaction = tx
});
snapshot.UnspentCoins.Add(tx.Hash, new UnspentCoinState
{
Items = Enumerable.Repeat(CoinState.Confirmed, tx.Outputs.Length).ToArray()
});
foreach (TransactionOutput output in tx.Outputs)
{
AccountState account = snapshot.Accounts.GetAndChange(output.ScriptHash, () => new AccountState(output.ScriptHash));
if (account.Balances.ContainsKey(output.AssetId))
account.Balances[output.AssetId] += output.Value;
else
account.Balances[output.AssetId] = output.Value;
if (output.AssetId.Equals(GoverningToken.Hash) && account.Votes.Length > 0)
{
foreach (ECPoint pubkey in account.Votes)
snapshot.Validators.GetAndChange(pubkey, () => new ValidatorState(pubkey)).Votes += output.Value;
snapshot.ValidatorsCount.GetAndChange().Votes[account.Votes.Length - 1] += output.Value;
}
}
// Iterate all tx inputs grouped together by common input transactions.
foreach (var group in tx.Inputs.GroupBy(p => p.PrevHash))
{
// Get the transaction state in common for the group of transactions to process.
TransactionState tx_prev = snapshot.Transactions[group.Key];
// For each input from this transaction.
foreach (CoinReference input in group)
{
// Mark the UTXO for the input as spent
snapshot.UnspentCoins.GetAndChange(input.PrevHash).Items[input.PrevIndex] |= CoinState.Spent;
TransactionOutput out_prev;
try
{
// Get the output from the the previous transaction that is now being spent.
out_prev = tx_prev.Transaction.Outputs[input.PrevIndex];
}
catch (Exception ex)
{
logger.Fatal($"Can't find the output being spent from tx hash: {input.PrevHash} index: {input.PrevIndex}{Environment.NewLine}{ex}");
// This really should never happen or something is bad wrong!
throw;
}
// Get the account for the address that owned the output being spent.
AccountState account = snapshot.Accounts.GetAndChange(out_prev.ScriptHash);
// If governing token was spent
if (out_prev.AssetId.Equals(GoverningToken.Hash))
{
// Keep track when this transaction input was spent to be able to calculate claims.
snapshot.SpentCoins.GetAndChange(input.PrevHash, () => new SpentCoinState
{
TransactionHash = input.PrevHash,
TransactionHeight = tx_prev.BlockIndex,
Items = new Dictionary<ushort, uint>()
}).Items.Add(input.PrevIndex, block.Index);
if (account.Votes.Length > 0)
{
// Reduce number of votes selected validators have due to the voter's reduced
// governing token blance.
foreach (ECPoint pubkey in account.Votes)
{
ValidatorState validator = snapshot.Validators.GetAndChange(pubkey);
validator.Votes -= out_prev.Value;
if (!validator.Registered && validator.Votes.Equals(Fixed8.Zero))
snapshot.Validators.Delete(pubkey);
}
snapshot.ValidatorsCount.GetAndChange().Votes[account.Votes.Length - 1] -= out_prev.Value;
}
}
try
{
// Reduce the balance of the asset by the account that owned the asset being spent
// by the the number of units of the asset that were spent.
account.Balances[out_prev.AssetId] -= out_prev.Value;
}
catch (Exception ex)
{
logger.Fatal($"Account ({account.ScriptHash.ToAddress()} = {account.ScriptHash}) doesn't have a balance for asset {out_prev.AssetId} {out_prev.Value} Block: {block.Index}{Environment.NewLine}{ex}" );
// This really should never happen or something is bad wrong!
throw;
}
}
}
...
We can't perform SETITEM
on ByteArray
, but maybe we can perform PICKITEM
on it.
Do you think so? @igormcoelho @shargon @lightszero @dicarlo2
Guys, do you think that's ok to change line
neo-vm/src/neo-vm/ExecutionEngine.cs
Line 588 in ab5f939
https://github.com/neo-project/neo-vm/blob/master/src/neo-vm/ExecutionEngine.cs#L618-L619
I can make the PR, just wondering if that's too crazy :)
how to use it in vm ware project?
Perhaps I'm unclear on exactly how HASKEY
should work with Array
StackItems, but I do not think this line is correct:
https://github.com/neo-project/neo-vm/blob/master/src/neo-vm/ExecutionEngine.cs#L890
Should it be something like this instead:
case VMArray array:
int index = array.IndexOf(key);
if (index < 0)
{
State |= VMState.FAULT;
return;
}
context.EvaluationStack.Push(index < array.Count);
break;
I propose that we adopt an extended multisig on neo 2.x and 3, that allow verifying witnesses as an alternative to pubkeys. We can validate size, if 20 bytes then getmessage and verify, otherwise must be pubkey.
This simple change allows many.important elaborate changes, for example, consensus nodes having surrogate backups that interoperate on network.
I think this is simple to change, but opinions are very appreciated.
Example:
push2 (limit)
push3 (count)
push20 ...
push33 ...
push20 ...
Lines 640 to 649 in 58b7a3a
Still remains
This is a minimal example to show the issue of inconsistent returns for BigIntegers.
I came across this while making a modification to a NEP5 'balanceOf' implementation where wallets are expecting a ByteArray return for the value.
/*
test1: Evaluation Stack: [{"type":"Array","value":[]},{"type":"ByteArray","value":"40420f"}]
test2: Evaluation Stack: [{"type":"Array","value":[]},{"type":"Integer","value":"999900"}]
test3: Evaluation Stack: [{"type":"Array","value":[]},{"type":"Integer","value":"999900"}]
test4: Evaluation Stack: [{"type":"Array","value":[]},{"type":"ByteArray","value":"dc410f"}]
*/
using Neo.SmartContract.Framework;
using Neo.SmartContract.Framework.Services.Neo;
using System.Numerics;
namespace Neo.SmartContract
{
public class Contract1 : Framework.SmartContract
{
public static object Main(string operation, params object[] args)
{
switch (operation)
{
case "test1":
return Test1();
case "test2":
return Test2();
case "test3":
return Test3();
case "test4":
return Test4();
}
return false;
}
public static BigInteger Test1()
{
BigInteger blah = 1000000;
return blah;
}
public static BigInteger Test2()
{
BigInteger blah = 1000000;
BigInteger blah2 = blah - 100;
return blah2;
}
public static byte[] Test3()
{
BigInteger blah = 1000000;
BigInteger blah2 = blah - 100;
return blah2.AsByteArray();
}
public static BigInteger Test4()
{
BigInteger blah = 1000000;
BigInteger blah2 = blah - 100;
Storage.Put(Storage.CurrentContext, "asdf", blah2);
return Storage.Get(Storage.CurrentContext, "asdf").AsBigInteger();
}
}
}
We propose the implementation of a modular exponentiation opcode for NeoVM.
The reason is that some dapp may require implementing RSA cryptography (which is our case on SciChain) and this is also fundamental for other number theory implementations, including zero-knowledge.
Currently, BigIntegers are limited to 32-bytes, but modular exponentiation may require a bit more than this, so one solution is to threat inputs and outputs as bytearrays (currently limited to 1024 bytes).
The initial proposal for MODPOW is to calculate: x = mul*base^{exponent} mod p
Values base
, exponent
, p
and mul
could be possibly passed in two options: (i) as an array [base
, exponent
, p
, mul
]; (ii) four independent elements taken from stack.
I prefer option (i), and it would be nice to allow mul
to be optional and naturally assume value 1
.
If exponent
is negative, it would be nice to have the calculation of the modular inverse.
All elements would be converted to BigInteger for the calculation (from bytearray), and the final result would be returned as bytearray (thus, not strictly violating BigInteger limits on NeoVM).
On C#, BigInteger ModPow could do the exponentiation and ModularInverse function could do the inverse.
The mul
could be eliminated by replacing it by two quadratic exponents with a subtraction and division (as Ethereum did on EIP 198), however that would require us to allow subtraction and division of huge numbers too (over BigInteger limit). We need to see if this mul
already solves most general problems involving modular exp applications.
To establish a fair price for the operation it would be nice to have some time experiments and see the actual impact, but it won't be more costly than VERIFY itself.
MODPOW opcode could possibly have opcode 0xb0
, although this is a discussion for later.
MODPOW could be acompanied later with another interesting operation which is scalar multiplication for elliptic curve points (possibly opcode 0xb1
). This would allow us to implement CHECKSIG
, CHECKMULTISIG
and VERIFY
directly on NeoVM, allowing users to switch cryptographic curves if they desire.
It would be a pleasure to have some voices of the crypto specialists in the community, in order to ensure a nice proposal (we can also create a NEP for this, if necessary).
Hello,
Is there any chance someone could document the OpCodes?
I have an additional question, if someone could answer, that would be great.
Why does the GoverningToken uses a PUSHT instruction and the UtilityToken uses a PUSHR instruction?
These are the transactions in the genesis block.
public static readonly RegisterTransaction GoverningToken = new RegisterTransaction
{
AssetType = AssetType.GoverningToken,
Name = "[{\"lang\":\"zh-CN\",\"name\":\"小蚁股\"},{\"lang\":\"en\",\"name\":\"AntShare\"}]",
Amount = Fixed8.FromDecimal(100000000),
Precision = 0,
Owner = ECCurve.Secp256r1.Infinity,
Admin = (new[] { (byte)OpCode.**PUSHT** }).ToScriptHash(),
Attributes = new TransactionAttribute[0],
Inputs = new CoinReference[0],
Outputs = new TransactionOutput[0],
Scripts = new Witness[0]
};
public static readonly RegisterTransaction UtilityToken = new RegisterTransaction
{
AssetType = AssetType.UtilityToken,
Name = "[{\"lang\":\"zh-CN\",\"name\":\"小蚁币\"},{\"lang\":\"en\",\"name\":\"AntCoin\"}]",
Amount = Fixed8.FromDecimal(GenerationAmount.Sum(p => p) * DecrementInterval),
Precision = 8,
Owner = ECCurve.Secp256r1.Infinity,
Admin = (new[] { (byte)OpCode.**PUSHF** }).ToScriptHash(),
Attributes = new TransactionAttribute[0],
Inputs = new CoinReference[0],
Outputs = new TransactionOutput[0],
Scripts = new Witness[0]
};
thanks
I'm porting the recent VM changes to neo-python
's VM and noticed the following
This line is supposed to calculate the stack item count
neo-vm/src/neo-vm/ExecutionEngine.cs
Line 156 in 41724f6
Assume a setup as follows:
InvocationStack
with 1 ExecutionContext
ExecutionContext
has 1 EvaluationStack
with 0 itemsExecutionContext
has 1 AltStack
with 1 item of type Array
Array
holds 3 StackItem
s.I would think stackitem_count
to be 3
(the 3 items in the array), but it counts Array
as another item so it returns 4
. Is this intended or should we ignore the actual Array
and Map
types in the count and only count their elements?
An Address
is converted to a ScriptHash
when using neo-python, which means that the Address
cannot be stored as a string within Storage
.
Add ability to convert ScriptHash
back to an Address
.
I'm currently working on a TypeScript/JavaScript -> NEO VM compiler and one thing that would be really handy is a native hash table type. While it's certainly possible to implement a hash table on top of the existing array functionality, it would be very script-space inefficient and GAS inefficient, requiring many op codes to properly access and manipulate the hash table. To that end, I propose we add 2 new types to the VM:
MapStackItem
which functions similarly to the ArrayStackItem
DictStackItem
which functions similarly to the StructStackItem
(that is, cloning in the same places StructStackItem
is cloned).ArrayStackItem
or StructStackItem
MapStackItem
or a DictStackItem
ArrayStackItem
, StructStackItem
, MapStackItem
, or DictStackItem
.OpCode.NEWMAP
: Pushes a new MapStackItem
onto the stack. Note that unlike NEWARRAY
, an initial size is not set because it does not make sense in this context.OpCode.NEWDICT
: Pushes a new DictStackItem
onto the stack. Similarly, does not have an initial size.OpCode.PICKMAPITEM
: Pops a key item key
off the stack. Pops a map-like item map
off the stack. Pushes map[key]
onto the stack. If key
does not exist, sets state to FAULT
.OpCode.SETMAPITEM
: Pops a stack item value
off the stack. Clones the value if it is a StructStackItem
or DictStackItem
. Pops a key key
off the stack. Pops a map-like item map
off the stack. Sets map[key]
to value
.OpCode.REMOVEMAPITEM
: Pops a key item key
off the stack. Pops a map-like item map
off the stack. Deletes map[key]
. If key
does not exist, does nothing.OpCode.HASMAPKEY
: Pops a key item key
off the stack. Pops a map-like item map
off the stack. Pushes true
onto the stack if key
is present in map
, otherwise pushes false
.OpCode.PACKMAP
: Pops an integer item length
off the stack. Creates a MapStackItem
and sets the items by popping a key item key
off the stack and a stack item value
off the stack length
times.OpCode.UNPACKMAP
: Pops a map-like item map
off the stack. Unpacks map
onto the stack in the same fashion that OpCode.PACKMAP
expects.OpCode.ARRAYSIZE
: modified to get the size of a map-like item.OpCode.APPEND
, OpCode.SETITEM
: modified to additionally clone DictStackItems
.These are nice to have, but not necessary given the above operations. However, they would help make certain operations more efficient and easier to manage, in particular OpCode.MAPENTRIES
and OpCode.ZIPMAP
.
OpCode.MAPENTRIES
: Pops a map-like item map
off the stack. Pushes an ArrayStackItem
that is an array of tuple ArrayStackItems
with [key, value]
onto the stack. I.e. Array<[key, value]
.OpCode.MAPKEYS
: Pops a map-like item map
off the stack. Pushes an ArrayStackItem
of the keys of map
.OpCode.MAPVALUES
: Pops a map-like item map
off the stack. Pushes an ArrayStackItem
of the values of map
.OpCode.ZIPMAP
: Pops an array-like item keys
off the stack. Pops an array-like item values
off the stack. Zips them together and pushes a MapStackItem
onto the stack.OpCode.UNZIPMAP
. Pops a map-like item map
off the stack. Unzips the map in the same fashion that OpCode.ZIPMAP
expects.One minor detail is the implementation will need to add hashCode
functionality and a custom comparator in order to have StackItem
keys. equals
, which would also be required, is already implemented.
Hello,
I am using string.Concat in my smart contract.
internal static string Generate(string prefix, string entityName, byte[] publicKey)
{
return string.Concat(prefix, entityName, publicKey);
}
internal static string Generate(string prefix, string entityName, byte[] publicKey, BigInteger id)
{
return string.Concat(prefix, entityName, publicKey, id);
}
Here the second method fails as BigInteger is used.
internal static bool Create(byte[] publicKey, string entityName, string name, BigInteger dd = 2)
In the above line I was not able assign default parameter to BigInteger, in this case compiler shows error.
Later:
internal static string Generate(string prefix, string entityName, byte[] publicKey)
{
return string.Concat(prefix, entityName, publicKey);
}
internal static string Generate(string entityName, byte[] publicKey, BigInteger id)
{
return string.Concat(entityName, publicKey, id);
}
Removed one parameter from the method... Works good
People, I don't know if it's something we can improve on neo-vm, or here at the compiler, but I really think this code should work:
using Neo.SmartContract.Framework;
using Neo.SmartContract.Framework.Services.Neo;
using System;
using System.Numerics;
namespace NeoContract1 {
public class Contract1 : SmartContract {
public static byte[] Main() {
byte[] b = new byte[]{0x01, 0x02, 0x03, 0x04};
b[2] = 0x05;
return b;
}
}
}
I generates the following AVM 00c56b0401020304765255c4616c7566
, that means:
00 PUSH0 #An empty array of bytes is pushed onto the stack
c5 NEWARRAY #
6b TOALTSTACK # Puts the input onto the top of the alt stack. Removes it from the main stack.
04 PUSHBYTES4 01020304 #
76 DUP # Duplicates the top stack item.
52 PUSH2 # The number 2 is pushed onto the stack.
55 PUSH5 # The number 5 is pushed onto the stack.
c4 SETITEM # bytearray[2] = 5
61 NOP # Does nothing.
6c FROMALTSTACK # Puts the input onto the top of the main stack. Removes it from the alt stack.
75 DROP # Removes the top stack item.
66 RET #
Operation SETITEM naturally fails, because it's only currently allowed for arrays or maps (not bytearray). I think it should be allowed on bytearrays, since the alternative to this is to do lots of substrings and concats (like this ugly solution: https://github.com/NeoResearch/learning-examples/blob/master/NonTrivialFunctions.md).
The problem I see is that it is applying a DUP
because it's guessing ByteArray is a reference type, so collateral effects should apply "remotely", but that won't work on copy type bytearray. So, how can we fix this in a easier way? Allow SETITEM to update bytearray too (on neo-vm project) and make the compiler avoid the DUP in these cases (in this project)?
========
Just to mention that this code works:
object[] obj = new object[]{0x01, 0x02, 0x03, 0x04};
obj[2] = 0x05;
But since it works as native array, it pushes elements one by one, and it's quite hard to convert it to byte[] (requires a full loop and many concats). So, a direct solution for byte[] would be nice, perhaps avoiding generating the DUP and allowing SETITEM to push byte[] back to stack after updating it.
Moved from: https://github.com/neo-project/neo-compiler/issues/133
I am interested in exploring using this VM in an internal project at Google, but I know from experience that our lawyers will push back pretty hard if I try to import MIT licensed code, especially code in a hot space where there are lots of patents being filed. The MIT license does not grant us any license to patents that might be filed by the authors of this code, and I've been told by lawyers that this creates an unacceptable risk before. However, IANAL, so I could be wrong...
Any chance of re-licensing or dual-licensing to something that will give potential users more confidence that they can use this code without getting sued?
Moved from this issue: neo-project/neo#140
Add ability to remove item from list.
The source code for Neo-VM and Neo is not consistent.
For example:
1, EvaluationStack is defined in the ExecutionContext class of Neo-VM, but it is used in the EvaluationStack class of Neo.
2, the loadScript function prototype defined in Neo-VM is “public ExecutionContext LoadScript (byte[] script, int rvcount = -1)”, and the call in Neo is engine.LoadScript(script, false)
3, and there are many other errors.
Please upload the latest stable version of NEO's source code to Github, thank you!
Let me start by explaining my interpretation of the function named GetCallingScriptHash and perhaps we can already stop there if that's wrong.
When I think of GetCallingScriptHash
then I believe that it will return the script hash of whoever is calling the smart contract this function is used in. 2 examples of that:
wallet 1
calls Contract A
, and Contract A
uses GetCallingScriptHash
-> I expect the script hash of Wallet 1
.
Contract A
DynamicAppCalls Contract B
, and Contract B
uses GetCallingScriptHash
-> I expect it to return the script_hash of Contract A
.
If that is the expected behaviour then GetCallingScriptHash
is currently limited to be called from the Main
function of the dynamically called contract. The question is; is this intentional, a bug or something that needs enhancement?
Let me elaborate on that with an example
Contract A:
def Main():
dynamicappcall -> contract B
Contract B:
def Main()
function_x()
def function_x:
GetCallingScriptHash()
The invocation stack frames are as follows;
0 -> the argument list provided on the console to the contract
1 -> Contract_A.Main
2 -> Contract_B.Main
3 -> Contract_B.function_x
Now; GetCallingScriptHash
does InvocationStack.peek(1)
(or basically takes the second last item of the invocation stack). That means it will return the script_hash of frame [2] which is Contract_B
, not Contract_A
(frame [1]) as the function name suggests.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.