self-explanatory...

The above contract is using https://7bshc4k5qfst26ewoutkb7ercnxaulkpv2an4wt5kunpq5jbqgvq.arweave.net/-GRxcV2BZT14lnUmoPyRE24KLU-ugN5afVUa-HUhgas source code. MemCached client produces one difference in comparison to the non-cached version and v1 SDK:

The issue might be with the "RandomHolder" function and line const r = new Prando_es_default(SmartWeave.transaction.id).next();

Needs further investigation.

calling contract.readState(txId, blockHeight) should not trigger call to arweave.networkInfo().

Also - verify how the deepCopy in https://github.com/redstone-finance/redstone-smartcontracts/blob/main/src/core/modules/impl/ContractHandlerApi.ts#L70 affects the performance - it is basically required by only one contract - ArweaveTeam/SmartWeave#92

The question here is - all of it, or simply some parts? And how to keep it in sync in all places (github, ArWiki, etc.)

https://gateway.redstone.finance - works fine
https://gateway.redstone.finance/ - results in 404 error.

Current version of the redstone-smartweave package does not play well with webpack v5+.

Using the latest create-react-app, generate a new sample project.

npm i arweave
npm i redstone-smartweave

add the following code...

const redstone = SmartWeaveWebFactory.memCached(arweave);
const ctr = redstone.contract("t9T7DIOGxx4VWXoCEeYYarFYeERTpWIC1V3y-BPZgKE");
LoggerFactory.INST.logLevel("error");
ctr.readState().then(r => console.log(r));

Observe as node dependency errors ensue...

Apparently there is a way to structure the package.json in the package to better support targeting the browser platform by default.

webpack/webpack#4674

^ provided here for context.

Expected Result:

Including redstone-smartweave should not introduce compile errors when included in a browser/web project.

Issue raised by Pianity.
In some environments the rss memory usage grows to very large values (it does not happen on my M1 though).
This is probably related to nodejs/node#40828

Removing the v8's serialization (https://github.com/redstone-finance/redstone-smartcontracts/pull/84/files#diff-5acab41990585fa68ae01ff7b2e2915628d19237840c3061b31fcadc1c0f09b8) fixed the memory leak issues, but at a cost of performance - therefore a better solution needs to be developed.

Currently it might be hard to understand/verify what exactly is happening during the state evaluation for contracts that heavily communicate with other contracts (and/or are utilizing the Foreign Call Protocol) - because of the "flat" structure of the logs.

A full call report should be generated (in a tree-like structure, with function names and parameters being used) and returned as another product of the state evaluation (next to state and validity report).

Note: validity should be probably also included - as currently it returns values only for the "root" contract.

The above contract currently evaluates for about 18 minutes on my M1. Still better than V1 SDK (where execution time approaches infinity ;-)), but certainly some further optimization can be applied in this case.

FCP - Foreign Call Protocol.

If I'm inside the interactions loop for contract A - let's say at interaction 50/100 - and this interaction calls contract B - and then this contract B calls contract A somewhere in the process again (very typical for FCP..) - executes it...and finally we return to the original execution of the contract A - it is moving in the loop to the 51/100 interaction - but it's not re-checking whether the result hasn't been already added to cache (as a side product of the execution of the 50th. interaction)

The read from the cache is currently done only for the "main" readState call.
Probably sth like this should be added to the interactions loop:

I just did import * as SmartWeaveSdk from 'redstone-smartweave' as in the docs and I get this error:

...\node_modules\redstone-isomorphic\npm-node.js:14
        global.Response = realFetch.Response;
                                    ^

TypeError: Cannot read property 'Response' of undefined

https://www.npmjs.com/package/fast-copy a blazingly fast 🤮 blah blah..

But it is faster than lodash/clondeep - for the Pianity contract (SJ3l7474UHh3Dw6dWVT1bzsJ-8JvOewtGoDdOecWIZo) evaluation time goes down from ~50s to ~28s

Added as an option in EvaluationOptions - useFastCopy - for now defaults to false, as I need to perform more regression tests

For example - TsLoggerFactory or BsonFileBlockHeightSwCache - which turned out to be issues for projects that are using the SDK in a web environment and are not properly utilizing the tree-shaking features of the bundlers like webpack or esbuild.

Currenty the TsLogFactory has been completely removed from all the static refrences in the code (eg. re-exports in the index.ts files), and BsonFileBlockHeightSwCache is used only in SmartWeaveNodeFactory.

Recent fix (PR #108) allowed to switch off by default caching of the state for each evaluated interaction.

Currently - to update state in cache for each evaluated interaction - updateCacheForEachInteraction flag in EvaluationOptions has to be set to true (previously true was the default value).

By default, the state is currently stored in cache:

1. before making foreign call to other contract
2. after evaluating all of the contract's interactions

But this may introduce an issue if RedStoneGatewayInteractionsLoader with notConfirmed mode is being used.

If given contract does not make any foreign calls and it happens to have any notConfirmed interactions - it may happen that state won't be stored in cache at all - because SDK prevents from caching interactions that are not confirmed (https://github.com/redstone-finance/redstone-smartcontracts/blob/main/src/core/modules/impl/CacheableStateEvaluator.ts#L203).
If the notConfirmed interaction was the last one evaluated, the CacheableStateEvaluator.onStateEvaluated hook won't save the state in cache.

In order to fix this, we need to track (during the state evaluation) the last confirmed interaction - and use this interaction as an input to CacheableStateEvaluator.onStateEvaluated hook.

Implement InteractionsLoader interface using the redstone gateway (https://github.com/redstone-finance/redstone-sw-gateway) REST API.
Constructor params:

1. base url (currently https://d1o5nlqr4okus2.cloudfront.net/)
2. [optional param] skipOrphans (default false - to be backwards compatible with current behaviour - we'll ofc. encourage projects to switch this to "true" - and maybe at some point set "true" as default).

Implementation:

1. Rename ContractInteractionsLoader to ArweaveGatewayInteractionsLoader
2. New class should be named RedstoneGatewayInteractionsLoader
3. use interactions gw endpoint - eg. <baseUrl>/gateway/interactions?contractId=<contractId>&from=<fromBlockHeight>&to=<toBlockHeight>&page=<current_page>
4. if skipOrphans is set, add param confirmationStatus=confirmed
5. interactions are loaded in pages - max. paging.limit per page. Use paging object from the response to iterate over pages.

{"paging":{"total":"9796","limit":500,"items":500,"page":1,"pages":20}

6. implement in a separate branch asiaziola/redstone-gateway
7. unit tests - Jest framework, mock the API responses.
8. regression tests - TBD
9. SmartWeave instance configuration - TBD

https://github.com/redstone-finance/redstone-smartcontracts/blob/8b61a4b6897056ce2eb1fc77c6600b14a9f83853/src/core/modules/impl/DefaultCreateContract.ts#L5

Trying to bundle this line using Vite breaks, since __filename i a nodejs feature, it breaks in the browser.

State should be cached per block height and per transaction id (instead of current caching per block height only - which is fine if you want to get "the last state for given block height", but not enough to load state for the exact transaction - which is required by both FCP optimizations and new writes between contracts).

Implementing this will allow to remove the "intermediary" cache introduced in #19 .

PST contracts with ~1100 interactions.

State evaluation:
Rust: 60-70ms https://sonar.redstone.tools/#/app/contract/SpTC4JuzZ2JDXR-lar2SbI2yJOxPgwi504BICe0uf1g
AS: 350-400ms https://sonar.redstone.tools/#/app/contract/_IHQHkZrZfB3lN69Hw3xTRcHv2cBiNgh1HG1WENydP4
Go: 120s .... https://sonar.redstone.tools/#/app/contract/QkVjYoiCq4dvru8z6laZmrL55vKskwbqyrAwGSn4cRQ

Some interactions for the Go evaluate for very long (~250ms) - it started from 600th interaction and then at about 900th returned to 1ms per interaction.

Needs investigation. Start with removing state clone before each action https://github.com/redstone-finance/redstone-smartcontracts-wasm-templates/blob/main/go/src/impl/contract.go#L31

The state cache should be checked before loading interactions - and only missing interactions should be loaded (ie. those between block height for highest cached state and requested block height).
This will allow to remove the need of storing a separate interactions cache.

Side note: It is still highly recommended to have a separate cache for interactions (otherwise - with only a state cache - there's a single point of failure - ie. if for some reason we loose the cached state, it might be very difficult to evaluate it from scratch (eg. loading all the transactions from Arweave might be problematic for contracts with many interactions)).
If - on the other hand - we have our interactions cached separately - it should be fairly easy to re-evaluate the whole state from scratch.

I use filecache ways to sync smc,
and I remove MemBlockHeightSwCache
and get this error
it is normally?

In case of "viewState" ("interactRead" from V1) - after properly evaluating the state (with the evolve plugin) - the last interaction transaction is being performed with the original contract source (and not the evolved one).

This might affect contracts for which the contract's method called by the "interactRead/viewState" has changed after "evolve".

It does not affect the PST's 'balance' method.

getting this message in my cli while interacting with smartweave. This makes. This is creating issue in case a frontend page trying to fetch data from a backend reading a contract, which is giving timeout error. And this error comes randomly. While the page shows code 503, the sdk keeps on functioning in the backend cli.

Should be implemented as a separate library with an ExecutorFactory implementation.

First idea worth checking out is the https://github.com/laverdet/isolated-vm - but of course this will work only in a node env.

Other:
https://2021.esec-fse.org/details/fse-2021-papers/46/JSISOLATE-Lightweight-In-Browser-JavaScript-Isolation
https://www.semanticscholar.org/paper/TrustJS%3A-Trusted-Client-side-Execution-of-Goltzsche-Wulf/03a79b989b9852b8bf4bb77c54bf8d5124efeba6

or use WASM?
https://webassembly.org/docs/security/
https://developer.mozilla.org/pl/docs/WebAssembly
https://engineering.q42.nl/webassembly/ - very good analysis!
https://www.assemblyscript.org/
https://developer.mozilla.org/en-US/docs/WebAssembly/Using_the_JavaScript_API

One of the ways to make an Arweave app fully decentralized is to run it all on client machine (i.e. no backend). To connect to Arweave network and smart contracts directly from the browser, I need a browser js bundle.

There should be an option for the user to decide what is the order of execution of his interactions mined within one block.
Currently - if two or more transactions for given smart contract are mined within one block, the order of execution depends on the ordering of sha256(transactionId + blockHash) - because of the random nature of the transactionId, it cannot be guaranteed that the order of execution will be the same as the order in which the transactions were posted.

An example use case would be:

1. User creates an interaction that approves token transfer on token contract
2. User creates an interaction that stakes tokens on staking contract (which underneath calls the token's contract "transferFrom" - firstly verifying if there's enough allowance (so the transaction from point 1. should be executed as first))

Add an option to change a state of a contract from within other contract code.

Proposed solution:

1. Write a new interaction for the "calling" contract.

• perform dry-run with the new input
• fetch a log of all inner write calls performed during this dry-run (probably with the use of #21)
• for each registered inner write call - add a tag in the resulting interaction transaction:
  'interactWrite': 'calleeTxId'

2. Eval state of the callee contract:

• load all interactions (both "traditional" and the one with "interactWrite": "calleeTxId")
• sort the transactions
• for all the "interactWrite" transactions - load the calling contract and eval its state up to the given "interactWrite" transaction height - this effectively should cause the "innerWrite" to be called - and with the help of the state cache for the callee contract - update its current value.

Detailed description

Introduction

SmartWeave protocol currently natively does not support writes between contract - contracts can only read each others' state. This lack of interoperability is a big limitation for real-life applications - especially if you want to implement features like staking/vesting, disputes - or even a standard approve/transferFrom flow from ERC-20 tokens.

Some time ago a solution addressing this issue was proposed - Foreign Call Protocol (https://www.notion.so/Foreign-Call-Protocol-Specification-61e221e5118a40b980fcaade35a2a718).

This is a great and innovative idea that greatly enhances contracts usability, but we've identified some issues:

• Contract developers need to add FCP-specific code in the smart contract code and in its state. This causes the protocol code to be mixed with the contract's business logic. Additionally - any update or change in the protocol would probably require the code (and/or state) of all of the contracts that are utilizing the FCP, to be upgraded.
• In order to create a "write" operation between FCP-compatible contracts, users need to create two separate transactions:

1. invoke operation on Contract B to add entry in the foreignCalls state field of the Contract B
2. readOutbox operation on Contract A, that underneath reads Contract's B foreignCalls and "manually" calls Contract'a A handle function for each registered 'foreignCall'

We believe that writes between contracts should be implemented on the protocol level (ie. contract source code and its state should not contain any technical details of the internal calls) and that performing a write should not require creating multiple interactions.

Solution

1. Attach a new method to the SmartWeave global object (the one that is accessible from the contract's code) with signature:
   function write<Input = unknown>(contractId: String, input: Input): Promise<InteractionResult>
   This method allows to perform writes on other contracts.
   The caller of such call is always set to the txId of the calling contract - this allows the callee contract to decide whether call should be allowed.
   The method first evaluates the target (ie. specified by the contractTxId parameter) contract's state up to the "current" block height (ie. block height of the interaction that is calling the write method) and then applies the input (specified as the 2nd. parameter of the write method). The result is memoized in cache.

This has been implemented in the Contract.dryWriteFromTx() and ContractHandlerApi.assignWrite()

2. For each newly created interaction with given contract - perform a dry run and analyze the call report of the dry-run (feature introduced in #21). This should generate a list of all inner-calls between contracts.
   For each generated inner call - generate additional tag:
   {'interactWrite': contractTxId}- where contractTxId is the callee contract.

This has been implemented in the Contract.writeInteraction and InnerWritesEvaluator.eval().

3. For each state evaluation for a given contract ("Contract A"):

• load all "direct" interactions with the contract
• load all "internalWrite" interactions with the contract (search using the tag introduced in point 4)
• concat both type of transactions and sort them according to protocol specification (i.e. using the sortKey)
• for each interaction:

1. if it is a "direct" interaction - evaluate it according to current protocol specification
2. if it is an "internalWrite" interaction - load the contract specified in the "internalWrite" ("Contract B") tag and evaluate its state. This will cause the write (described in point 1.) method to be called. After evaluating the "Contract B" contract state - load the latest state of the "Contract A" from cache (it has been updated by the write method) and move to next interaction.

This has been implemented in the DefaultStateEvaluator.doReadState()

This method is also effective for nested writes between contracts.

Alternative solution

An alternative solution has been also considered. Instead of writing only a tag with id of the calling contract (point 3. from the description above), one could write the exact input (ie. function, parameters, etc) of the call. This would have an advantage of the increased performance (as we could evaluate the "Contract A" state without the need of evaluating the "Contract B" state in the process), but would ruin the "lazy-evaluation" idea of the protocol and reduce the safety of the solution - as the Contract A would need to fully trust the calling Contract B - if the Contract B would save some unsafe data in the input - there would be no way to exclude it from the execution.

createTx() overwrites user-provided Tag values with default values, including setting the 'App-Name' and 'App-Version' Tags to fixed default values. This makes using the 'redstone-smartcontract' package unacceptable for production applications.

..when used together with RedStone Gateway and "notCorrupted" mode.

Issue raised by the Pianity team.

Original repo that tries to reproduce the issue: https://github.com/ppedziwiatr/redstone-smartweave-cache-bug
The above example makes "concurrent" interactions writes to ArLocal, with block time set to 0.5s.
This results in many interactions being mined at the same block height.
Each interaction write is preceded with a dry-write. Each dry-write first reads current state - and stores it in cache. It then applies new input and verifies whether it will be successful.

Original issue: state returned by client used during the experiment differs from the result from a freshly created client (i.e. with no cached state) or when state is read using the original smartweave.js.

Original diagnosis: not all transactions were mined during the state reads after running original experiment. Either wait for the mining (5s is enough for an experiment with 100 interactions on my M1) or read the state at concrete block height - fixed in ppedziwiatr/redstone-smartweave-cache-bug@fd8a456

On my M1 this has fixed the issue, and with sleep, and/or reading at the same height for all clients - I was getting stable results, even for experiments with 1000 interactions.

However, the issue persists on a slower machine - e.g. it is reproducible on Dell Precision M4800 with i7-4810MQ cpu. (on M1 the experiment with 100 interactions takes ~12s. On i7-4810MQ the same experiment takes ~65-70s..)
The issue is that state returned by freshly created client and smartweave v1 are the same, but differ from result from already cached client.

Analysis in this case is very difficult ("concurrent" writes of the transactions) and each run gives slightly different output.
After enabling logging in the SDK and after many more experiments the diagnosis is:

Diagnosis

There is an issue with loading interactions from ArLocal's GQL endpoint during the experiment (i.e. when interactions are being created and dry-runs are being run). The root cause might be the caching issues on the ArLocal's GQL endpoint - or basically some delay between mining transactions by the ArLocal and when they are available/returned by the ArLocal's GQL endpoint.

Proof:

1. All experiments always run on a fresh ArLocal instance with a fresh db. The SDK instance created without caching the interactions, i.e.:

const smartweave = SmartWeaveNodeFactory
        .memCachedBased(arweave, 1)
        .setInteractionsLoader(new ArweaveGatewayInteractionsLoader(arweave))
        .build();

2. Example screenshot from the experiment (i.e. when transactions are being created and dry-writes are being called)
   

As noted on screenshot, at this point GQL endpoint returned wrong result when asked for interactions for height [55; 57] - i.e. it returned only 3 interactions, whereas it should return 10 (which I'll prove in next points). This causes that (in this case - for each run it happens at random/different moments) the SDK caches at block height 57 the wrong state (because it was evaluated only for 3 interactions, not 10).
From now on - each next state evaluation with this SDK instance will result in a wrong state being returned.

3. After the experiment has ended (and additional 5s timeout), running the state read on a fresh client for block height 56 results in loading 20 interactions and balance = 20:
   

4. The same, but run at block height 57 results in loading 30 interactions (if the GQL endpoint would behave the same, as during the experiment (point 2.), it would return 23 interactions). This also proves, that in fact there are 10 interactions between 56-57 (and not 3). In fact, all 10 interactions are at height 57 (next point)
   

5. additional proofs - from the graphql endpoint:
   read at height 55-57 (exactly the same as in point 2., where only 3 interactions were returned):
   

read at height 57:

Same result, all 10 (not 3, as GQL returned during the experiment) are at block height 57.

Summary

In case of this issue I don't see any fix from the SDK point of view. The SDK trusts that interactions returned from the gateway (ArLocal GQL endpoint in this case) are proper (i.e. proper amount with proper input data). It doesn't "know" that one given response might be flawed.

Those GQL endpoint caching issues also do occur on production arweave.net server (in fact, there's currently an issue being worked on - https://discord.com/channels/357957786904166400/812013044892172319/930980333741015060 ). That's why it is very risky to cache the state without any additional verification / consensus - as one have no guarantee, that result (i.e. interactions - which are the "input" for the whole state machine) returned by gateway's GQL endpoint is proper.

One workaround would be to periodically run state reads at the same given height and verify, whether the results are stable/the same.

Hello Redstone, this is Andres from Verto.exchange.

Overview:
Plugins are a way to introduced custom Javascript functions to the execution of smart weave contracts.

Details:
Plugins should be passed in a static call to a setter, for example:

import { setPlugins } from "smartweave";

setPlugins({
   helloWorld: () => console.log("hello world");
});

Or they can be passed in the readContract function.

export declare function readContract(arweave: Arweave, contractId: string, height?: number, returnValidity?: boolean, plugins?: any): Promise<any>

import { readContract } from "smartweave";

readContract(arweaveClient,
            contractId,
            undefined,
            true,
            {
                helloWorld: () => console.log("hello world");
            }
);

Accessing plugins
In order to access the plugins inside the contract, we can do:

SmartWeave.plugins[plugin_name]();

where SmartWeave is the global that all contract has, with a new object, which is call plugins, and contains the functions we passed in our object

SmartWeave.plugins['helloWorld']();

Security Flaws
A contract can override, or modify the plugins object. For this reason, this object should be frozen, and all its properties should also be frozen. A deep Object.freeze if you may.

cc @t8

Tags encoding for "dummy" transactions (used for viewState and dry-write functions) is inconsistent with encoding in real transactions - i.e. tags. in dummy transactions are encoded, tags in real transactions - decoded (that's how both arweave.net GQL and redstone gateway returns them).
https://github.com/ppedziwiatr/smartweave-tags-encoding

Axios is slooooow.
Undici is fast.

Better benchmarks - yay!
https://nodejs.medium.com/introducing-undici-4-1e321243e007
ArweaveTeam/arweave-js#134 (comment)

and verify, if it won't fuck-up the web env. (still wondering if we should keep on supporting the web target..it's somewhat pain)

abort handler should rethrow original abort cause.

Consider adding an Evolve plugin by default - and/or change API for configuring the executionContextModifiers of the StateEvaluator - as currently the dev exp. sucks - ie. in case you want to use custom cache for the DefaultStateEvaluator AND want to use the Evolve plugin - you need to configure from scratch the DefinitionLoader and ExecutorFactory.

migrate all the dry-run related features from the original SmartWeave SDK (mostly dry interact writes).
Probably should be kept in a separate module, in order no to clutter the core protocol interfaces.

Currently, when the internalWrites flag is set to true in the contract EvaluationOptions, the following error is received while deploying a new contract and writing an interaction immediately afterwards: Unable to retrieve tx <txId>. undefined. undefined.
This could be solved by using RedStone gateway and sequencer to deploy contracts, as it is currently done for contract interactions.
Proposition: add an option for bundled contract creation for these two functions:

• smartweave.createContract.deployFromSourceTx()
• smartweave.createContract.deploy()

Loading transaction data from arweave.net/tx/id is probably a bad idea (as it using the arweave.net cache).
Transactions should be loaded directly from peers.

Arweave discord question: https://discord.com/channels/357957786904166400/812013044892172319/944168233336987658

Issue raised in https://discord.com/channels/786251205008949258/816685554329583707/899946318951231488

The normalizeContractSource currently removes IIFE expressions (that may be added by bundlers like esbuild - when using "IIFE" bundle format) - but it should remove only the "main" IIFE expression, that wraps the whole contract (and not the "inner" ones, that may be within the contract source)

Example source that is causing the issue: https://nn553dl24zfsc6wowshdxz25pf5ynolkjh5xjat44wsipwnm4hda.arweave.net/a3vdjXrmSyF6zrSOO-ddeXuGuWpJ-3SCfOWkh9ms4cY
contract: BXCEBKTv-Fvan0m82aEi7njdnRsFDluFkfN8vrWG5FI

The same issue is on the current version of the original smartweave.js SDK.

regression: difference in evaluated state for contracts:
w27141UQGgrCFhkiw9tL7A0-qWMQjbapU3mq2TfI4Cg,
Qa9SzAuwJR6xZp3UiKzokKEoRnt_utJKjFjTaSR85Xw,
38TR3D8BxlPTc89NOW67IkQQUPR8jDLaJNdYv-4wWfM

when cache is updated for onContractCall hook.

turns out Knex cannot translate properly group by queries between different db engines...duh.
https://discord.com/channels/786251205008949258/887686486508568586/931424019838816277

$ ts-node -r dotenv/config ./scripts/redstone/read.knex.cache.ts
error: select "contract_id", "height", "state", max("height") from "states" group by "contract_id" order by "height" desc - column "states.height" must appear in the GROUP BY clause or be used in an aggregate function
    at Parser.parseErrorMessage (ar-contracts/node_modules/pg-protocol/src/parser.ts:369:69)
    at Parser.handlePacket (ar-contracts/node_modules/pg-protocol/src/parser.ts:188:21)
    at Parser.parse (ar-contracts/node_modules/pg-protocol/src/parser.ts:103:30)
    at Socket.<anonymous> (ar-contracts/node_modules/pg-protocol/src/index.ts:7:48)
    at Socket.emit (node:events:390:28)
    at Socket.emit (node:domain:475:12)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:199:23) {
  length: 176,
  severity: 'ERROR',
  code: '42803',
  detail: undefined,
  hint: undefined,
  position: '23',
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'parse_agg.c',
  line: '1419',
  routine: 'check_ungrouped_columns_walker'
}

Regression tests are super important, because they give us a high degree of confidence, that the state evaluated by the SDK is proper...BUT the current performance is killing me. On my M1, regression of all the ~110 verified contracts runs for about 12m, on GH actions it takes about 30m?

The main issue here are the calls to Arweave GQL endpoint (or rather slow response times).

Now, I wonder if maybe we should migrate these tests to our RedStone Gateway - this would speed them up significantly and would still test the SDK implementation. RedStone Gateway has its own set of regression tests, so this shouldn't affect the overall testing "quality".

@asiaziola , what do you think?

Recently a new APIs have beed added - dedicated for interacting with PST (and for evolving contract's source code).
It would be good to mention them in the docs (or create a separate - dedicated Readme?)

There seems to be some new bundledIn property, which - when is set - tells, that the transaction is a bundle - and should be skipped in a same way, as transactions with parent property are being skipped (https://github.com/ArweaveTeam/SmartWeave/blob/master/src/contract-read.ts#L206)

https://discord.com/channels/357957786904166400/756557551234973696/920771718396653659

import knex from "knex";
import { LoggerFactory, SmartWeaveNodeFactory } from "redstone-smartweave";
import Client from "../utils/init";
import fs from 'fs'

const knexConfig = knex({
    client: 'pg',
    connection: 'postgres://redstone:redstone@localhost:5432/smartweave',
    searchPath: ['knex', 'public'],
    useNullAsDefault: true
})

const wallet = JSON.parse(fs.readFileSync("./wallet.json") as unknown as string);

async function KnexCached() {
    const smartweave = await SmartWeaveNodeFactory.knexCached(Client, knexConfig)
    LoggerFactory.INST.logLevel("debug", "Read contract interactions")

    const contractTxId = "-8A6RexFkpfWwuyVO98wzSFZh0d6VJuI-buTJvlwOJQ"
    const contract = smartweave.contract(contractTxId).connect(wallet)
    const { state, validity } = await contract.readState()
    
    console.log(state);
    console.log(validity);
}

KnexCached().catch(e => console.log(e))

I'm trying to read contract state using knex, postgres. I'm getting below error Can you please tell anything wrong in code or setup?

warning ../package.json: No license field
$ ts-node -r dotenv/config ./scripts/redstone/read.knex.cache.ts
error: select "contract_id", "height", "state", max("height") from "states" group by "contract_id" order by "height" desc - column "states.height" must appear in the GROUP BY clause or be used in an aggregate function
    at Parser.parseErrorMessage (ar-contracts/node_modules/pg-protocol/src/parser.ts:369:69)
    at Parser.handlePacket (ar-contracts/node_modules/pg-protocol/src/parser.ts:188:21)
    at Parser.parse (ar-contracts/node_modules/pg-protocol/src/parser.ts:103:30)
    at Socket.<anonymous> (ar-contracts/node_modules/pg-protocol/src/index.ts:7:48)
    at Socket.emit (node:events:390:28)
    at Socket.emit (node:domain:475:12)
    at addChunk (node:internal/streams/readable:315:12)
    at readableAddChunk (node:internal/streams/readable:289:9)
    at Socket.Readable.push (node:internal/streams/readable:228:10)
    at TCP.onStreamRead (node:internal/stream_base_commons:199:23) {
  length: 176,
  severity: 'ERROR',
  code: '42803',
  detail: undefined,
  hint: undefined,
  position: '23',
  internalPosition: undefined,
  internalQuery: undefined,
  where: undefined,
  schema: undefined,
  table: undefined,
  column: undefined,
  dataType: undefined,
  constraint: undefined,
  file: 'parse_agg.c',
  line: '1419',
  routine: 'check_ungrouped_columns_walker'
}

The transactions queried by graphql on the chain do not check whether the transactions have been packaged on the chain, this should be a graphql bug, but I think redstone-smartcontract should check that the block field in the transactions returned by graphql cannot be null.

My discovery process:
First I am using our own implementation of the arweave graphql interface to provide queries for redstone-smartcontract.
I tested using VRT: usjm4PCxUd5mtaon7zc97-dt-3qf67yPyqgzLnLqk5A and found that the redstone-smartcontract connection to arweave.net/graphql executed more validity txIds than using everFinance's graphql.
Then I found out that the extra validity txId is not available in the chain including the viewblock. So I used graphql to query the tx directly, and found that the block returned for these transactions was null, meaning that none of the transactions were packaged by the block.

Test Data:
VRT contract: usjm4PCxUd5mtaon7zc97-dt-3qf67yPyqgzLnLqk5A
arweave graphql test code: https://github.com/redstone-finance/redstone-smartcontracts-examples/blob/main/src/verto-contract-example.ts

everFinance graphql interface is not currently open, The following is the validaity txId returned by arweave grphql that is not chained:

 "HSomvmj3Ah644qFmz2b-k9yEgi-_ZYxEuk14j3p1PgI":true,
  "25FNTSfxf9IlEDP7L5DVv0QlHhGg3mIXsufIAXrNWro":true,
  "EXxQSzRi3yIC2YQwEkZO-AmoqHTnL6zYgW9PCm26cGk":true,

  "7XQfPu9FFww_qumtlw6Aw_F_ausmOyvhTGIayIJHnJw":true,
  "HgKANz3kmk2j9BbVWMcisuJx4W8cJ2g7eAYKJ2TEx-U":true,
  "FqwBfSFMWPBfijMWexPoa6sUonTWqLXpQS2NvT7D40g":true,
  "HupMiMiALM8xj5xyBniEA-Y951Qn1VFih7yNrzqlISM":true,
  "A_KU8-hKvBKAfsy6XeJCIP9TkjpqIXqOoiIL_iCn4Vw":true,
  "n9_adT2UdLlSIicim_fEw1y7DnpyftmQsMNggyq12LU":true,

You can verify this with the following graphql:
`url: https://arweave.net/graphql
query:

query{
  transaction(id:"25FNTSfxf9IlEDP7L5DVv0QlHhGg3mIXsufIAXrNWro"){
        id
        owner {
          address
        }
        recipient
        tags {
          name
          value
        }
        block {
          height
          id
          timestamp
        }
        fee {
          winston
        }
        quantity {
          winston
          ar
        }
        bundledIn {
          id
        }
      }
    }

Request from "noom" on Discord - https://discord.com/channels/786251205008949258/816685554329583707/919950479046348863

Implementation - add an optional "caller" param to Contract.dryWrite()

because of #97 - we're temporarily switching back to loading transactions with arweave-js (which is using axios underneath = slooow).

The final solution should be

1. try to load tx and tx data with our ArweaveWrapper (that is using undici and loads tx from arweave.net cache)
2. if it fails - fallback to loading with arweave-js.