With the basic flow in place (using the compiled abi from the computable repo in this js repo), continue to define Typescript types and flush out the spec suite for this contract

Its likely that our biggest use case will be consumer projects that want to import our files in their own TS projects (as our demos will). Export them from the top level so that import statements are simply:

import { Foo } from 'computable'

Reflect the forming rules and expectations.

The datamined protocol intermingles on-chain and off-chain computation. datacoin's will have to live on-chain. Infrastructure for datacoin will have to live on-chain as well, such as Wallet. However ComputeNode, StorageNode and Validator contracts (see #17), will need to perform off-chain computation. StorageNode will be responsible for storing and transferring large datasets, while ComputeNode will need to compute summary statistics or perform machine learning on them. For machine learning, being able to use standard python libraries like numpy or deephem or tensorflow will make things much easier.

It's not yet clear how to co-minggle on-chain and off-chain computation in a protocol. Potential references that might be worth following up on:

• IPFS/Filecoin: Storage is done off-chain. How does this interact with on-chain tokens/contracts?
• Truebit/Golem: Compute is done off-chain. How does this interact with on-chain tokens/contracts?

The recently added (#22) examples/wallets.py file runs transactions on a wallet contract on a locally created geth node. However, running these transactions appears to take some amount of time, say up to 15 seconds for a few withdraw/deposit commands, since the transactions have to be mined by geth.

It's not clear whether larger economies can be implemented and run on a local geth node. In that case, might need to default to an in-memory tester chain for debugging/prototyping larger systems.

so that it can be imported via computable/dist/helpers

A number of unsavory deprecation warnings crop up from populus, eth_utils and other under-the-hood packages. These are upstream warnings we don't have much direct control over. Find a way to suppress warnings:

  "The `compose` function has been deprecated and will be removed in a "
/home/rbharath/anaconda3/envs/datamined/lib/python3.6/site-packages/populus/project.py:235: DeprecationWarning: project.contracts_source_dir has been replaced by the plural, project.contracts_source_dirs which is an iterable of all source directories populus will search for contracts.  Please update your code accordingly as this API will be removed in a future release
  "project.contracts_source_dir has been replaced by the plural, "
/home/rbharath/anaconda3/envs/datamined/lib/python3.6/site-packages/populus/chain/tester.py:13: DeprecationWarning: currentProvider is deprecated in favor of `providers`, which is now a list
  self.rpc_methods = self.web3.currentProvider.rpc_methods
/home/rbharath/anaconda3/envs/datamined/lib/python3.6/site-packages/eth_abi/abi.py:92: DeprecationWarning: Automatic inference of hex encoded data has been deprecated. Please adjust your code to ensure that the data argument for `decode_single` is a byte string
  "Automatic inference of hex encoded data has been deprecated. "

Devs should have TSC compilation happening locally via either their editor or manually using the CLI. Let's also have travis run the compiler to be sure

Methods on the Market Token not inherited from the ERC20 super...

Not a huge fan (TSC is really all that is needed) as this is run by Palantir, and its just one more thing to maintain, but, it may be worth doing.

Timeboxing a cpl hours to see if I want it in.

Implementing the datamined protocol will require adding a number of new solidity contracts. This PR tracks the changes that will be needed to change our current solidity code into a functioning datamined prototype:

• ComputeNode contract for compute nodes.
• StorageNode contract for data storages.
• DataMiner contract for the users/miners
• Validator contract for the validating entries.
• DataCoin contract for the actual datacoins used in the network.

Each of these contracts will require associated tests as noted in #16.

The codebase for datamined is growing large enough that we should start running continuous integration tests. The simplest way to do this is likely through Travis CI.

After following the directions in the README, npm run test fails because it can't find EIP20.json. This is due to the the recent move from the EIP20 to ERC20 token in the computable repo.

#86 fixes this temporary problem but looking forward, might it be better to keep the computable submodule pinned to a separate, special branch within the computable repo which specifically exists to keep in sync with computable.js master and remove the git submodule update --remote line from the README instructions? This might avert this problem in the future. Let me know if I've misunderstood anything.

With some patters emerging, and direction becoming clearer it will soon be time for a quick audit of type usage. Newer TS versions have introduced some better advanced types like:

• Required
• Pick
• Omit
• Record

And I know that I could do a better job using Partial. I refuse to prematurely optimize, but this won't run the danger of being such.

One of the targets for the Computable 0.2.0 release is support for Recursive TCRs computablelabs/computable#64, that is, TCRs which contain TCRs as sub-entries. To first approximation, a recursive TCR can be implemented simply as a TCR of addresses of other TCRs.

We need to implement a spec that uses the js API to instantiate this type of TCR of TCRs and tests some basic properties of recursive TCRs

After resolving issue #56 with a switch to node 8.11.1, I unsurprisingly had to do an npm rebuild to get past

Error: The module '/Users/rstewart/src/computable.js/node_modules/scrypt/build/Release/scrypt.node'
was compiled against a different Node.js version using
NODE_MODULE_VERSION 64. This version of Node.js requires
NODE_MODULE_VERSION 57. Please try re-compiling or re-installing
the module (for instance, using `npm rebuild` or `npm install`).

When I then ran npm run test, I got

% npm run test

> @computablelabs/[email protected] test /Users/rstewart/src/computable.js
> jasmine-ts ./spec/**/*.spec.ts

Error: Cannot find module '../../../computable/build/contracts/Parameterizer.json'
    at Function.Module._resolveFilename (module.js:547:15)
    at Function.Module._load (module.js:474:25)
    at Module.require (module.js:596:17)
    at require (internal/module.js:11:18)
    at Object.<anonymous> (/Users/rstewart/src/computable.js/spec/computable/parameterizer/parameterizer.spec.ts:23:46)

The top level computable dir is empty and the spec/computable dir doesn't have a build dir. Is there anything else I need to do to get a clean build other than npm rebuild?

I don't want us to be maintaining 2 js environments. computable.js should be the sole place for all things computable js-related. Now that we know we can sub-module the contracts into here, move and convert (and create if absent) the client tests for the computable protocol .sol contracts.

Then remove them from computable (in a separate ticket).

With the basic flow in place (using the compiled abi from the computable repo in this js repo), continue to define Typescript types and flush out the spec suite for this contract

see dist/@types/ganache-cli

No declarations via NPM so we will type them out here then contribute it to the main repo when ready.

In #9, added in a simple crypto-economy. This economy suffers from a number of design flaws and limitations:

• The set of miners doesn't grow over time. Instead of instantiating a simple fixed set of miners, should rather have some type of random emission distribution that models the introduction of new miners into the network. (Poisson?)
• Each miner should bring new data onto the network sporadically. Need another emission distribution (Poisson?)
• There needs to be a decay distribution controlling the data mining rates. Are we decided on validators as the central mints of the protocol?
• It's starting to look like clients should know what their ledgers are for convenience.
• We should model currency buy-ins (where mined datacoins are swapped for BTC/ETH) since it's likely that the first buyers will enter the data economy that way rather than our current model (where miners are also buyers)
• New buyers should also be introduced randomly (Poisson?)

Adding some of these pieces will make the economy simulation somewhat more accurate.

Ledger contains pointers to where data is stored but the pointers themselves will require encryption so that only permissioned users receive access. One way I've thought about this is a multi-sig approach where data owners grant permissions by signing a data transaction with their private key for users that can uniquely countersign with their private key. Data is then pushed to permissioned user for computation. "Pushing" here also requires more thought. For example, data could be copied to a temporary storage location that the multi-sig transaction reveals for users to then grab (one-time before deletion), without revealing where original copies of data are located (and encrypted).

https://github.com/datamined/datamined/blob/0c805273deedd45ace097aea3148da1b05844706/examples/workflow.py#L19-L24

Why is the Parameterizer compiling correctly here and the ERC20 not?. Fix.

It's clear that all send type state-changing methods should return a "receipt" and that we can easily
provide helpers to work with that returned object.

This issue then has 2 parts

• return the TR from deploy
• create helper to retrieve address from returned TR (NOTE: there is already HOC.getAddress so if the returned TR doesn't contain the address [i think it does...] just use the method...)

Abstract away the web3 contract inside a "HOC" that exposes the methods with a more succinct syntax.

see dist/@types/solc

No declarations via NPM so we will type them out here then contribute it to the main repo when ready.

see #105

We anticipate that datamined will be used to store sensitive data. In the U.S, sensitive medical/healthcare data requires HIPAA compliance. While literature has emerged on HIPAA compliant architectures, it's not clear what a HIPAA compliant dApp would look like. Let's use this issue to track the changes/features that will be needed to bring datamined into HIPAA compliance.

Feel free to chime in if you have comments/thoughts on what HIPAA compliance means for dApps in general.

Test out the theory that we can maintain a single Typescript/Javascript environment, not only for computablejs - but for computable as well by "submodule-ing" the protocol repo's solidity contracts here.

Now that datamined economies can be created programmatically, we will need tools to instrument and study these economies. Here are some preliminary proposals for better instrumentation:

• Create an Economy class with an associated get_number_tokens() method that displays economy size.
• Add some simple plotting code to visualize economy sizes over time.

One of the declaration files in dist/@types that we have begun as there is no 'official' set of typings for it.

We'll contribute this back to the main @types repo when ready.

When I run npm install, I get an error message because node-gyp still does not support Python 3.

> node-gyp rebuild

gyp ERR! configure error 
gyp ERR! stack Error: Python executable "/Users/rstewart/anaconda3/bin/python" is v3.6.4, which is not supported by gyp.
gyp ERR! stack You can pass the --python switch to point to Python >= v2.5.0 & < 3.0.0.

According to this long issue thread, there is some hope for Python 3 support - nodejs/node-gyp#193

To force the usage of Python 2.7, I tried npm install --python=python2.7. That produced a different error message.

> node-gyp rebuild

  CXX(target) Release/obj.target/sha3/src/addon.o
../src/addon.cpp:59:36: error: no matching member function for call to 'NewInstance'
                        info.GetReturnValue().Set(cons->NewInstance(argc, argv));
                                                  ~~~~~~^~~~~~~~~~~
/Users/rstewart/.node-gyp/10.0.0/include/node/v8.h:3851:44: note: candidate function not viable: requires single argument 'context', but 2 arguments were provided
  V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(
                                           ^
/Users/rstewart/.node-gyp/10.0.0/include/node/v8.h:3848:44: note: candidate function not viable: requires 3 arguments, but 2 were provided
  V8_WARN_UNUSED_RESULT MaybeLocal<Object> NewInstance(
                                           ^
1 error generated.
make: *** [Release/obj.target/sha3/src/addon.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/usr/local/lib/node_modules/npm/node_modules/node-gyp/lib/build.js:258:23)

Any other magic incantations I should try?

The TCR Parameterizer needs a deployed token to function, so test that out and perhaps create a helper for later contract specs that will need it.

Can't think of why I did that -- prob a cpy/paste err...

The datamined.coins module now uses a simple ExampleWallet to track coins. This basic wallet should be migrated onto the blockchain sooner than later. This integration should probably have the following elements:

• A simple datacoin smart-contract.
• Perhaps a wallet smart-contract. Not yet sure if this is better done in solidity or in pure python.

The best Python ethereum development tool out there appears to be populus. https://github.com/pipermerriam/populus. Will experiment with populus and update this thread with notes.

Our higher-order contracts will take an opts argument to any method that requires a send. ATM it is being used for divining out the sender, but should expand to use any passed in gas and gasPrice args.

A simple change, but one to do before next test-net push.

The datacoin folder now has a simple set of solidity code for Wallets/Tokens/Crowdsales. There is also a dummy Greeter contract along with a simple test suite. Tests need to be added for the Wallet/Token/Crowdsale contracts.