beyond-blockchain / bbc1 Goto Github PK

API documents in readthedoc.io is something wrong (no detailed information is output). The documentation tool "sphinx" works fine in a local environment, but not in readthedoc.io environment.

New classes, BBcAssetRaw and BBcAssetHash, are introduced in py-bbclib v1.5.
The header version should be 3 to support these classes.

data_handler needs to be understand BBcAssetRaw and BBcAssetHash to create appropriate meta tables.

Currently, bbc_core checks whether the accessing transaction has been falsified or not. This means that the check is conducted passively. We probably need a kind of active check functionality.

Currently, ledger_subsystem supports anchoring to Ethereum only.
We plan to support anchoring to bitcoin, too.

In the case that a transaction to be inserted includes multiple BBcEvents and multiple asset_group_ids, the transaction should be registered to the ledgers of these asset_group_ids. However, the method insert_transaction() in bbc_core.py requires a parameter of asset_group_id, leading to registering just once for the asset_group_id.

test_bbc_ethereum.py and test_ledger_subsystem.py do not terminate geth process at exiting the program.

NAT traversal is definitely needed.

core/ethereum cannot be set up when "How to use" in README is followed.
It fails at "5. Deploy BBcAnchor smart contract".

This is because "genesis.json" and "populus.json" are wrongfully included in the release.
They need to be removed from the repository.

Also, in README, there need to be a new line before "5. Deploy BBcAnchor smart contract".

Owing to the python module (pure python implementation) of ECDSA, the processing speed is very slow. So, we will use an OpenSSL implementation. We will implement a wrapper for OpenSSL library because we cannot find a suitable python wrapper module for BBc-1.

MessagePack is used for object type asset body. Decoding a packed data by golang bbclib in python is OK and vice versa. However, the transaction digest calculated in python is sometimes different from that in golang because encoding rule is little bit different.

A value (char, short, int, long,,,) seems to introduce such an inter-platform gap. So, the object type asset body should be in {"string key": "string value"} type for inter-platform compatibility.

Error with "pyopenssl 18.0.0 has requirement cryptography>=2.2.1, but you'll have cryptography 2.1.4 which is incompatible." results in failure of pip installation.

Domain is currently not supported in the ledger subsystem.
Also, the ledger subsystem code contains redundant db functions copied from bbc_ledger.
(bbc_ledger may have a problem when tables with arbitrary indices are created in the auxiliary db.)

When testing some old code written for the past core <v1.2 (bbc1-lib-std and bbc1-lib-tokens)
by modifying to import bbclib as follows:
from bbc1.core.compat import bbclib

results in:
E AttributeError: 'BBcTransaction' object has no attribute 'pack'
during serialization.
(bbc1/core/bbclib.py:90: AttributeError)

In the packing method of BBcAsset (bbclib_asset.py), the "asset_file_digest" is packed with length value for data packing. However, for calculating "asset_id", asset_file_digest is packed without length value. This is confusing and inconsistent in terms of format design (it may be just a kind of bug). Even though it is not fatal issue, this should be fixed.

"asset_id" is just an identifier. So, adding length field in packing for calculation of asset_id would be OK.

In addition, asset_file_digest is truncated by id_length in some cases in bbclib_asset.py. It is also inappropriate and should be fixed.

macOS
Python 3.7.3
Flask 1.0.1
bbc1 1.3

For example, run below code.

from flask import Flask

from bbc1.core import bbc_app

app = Flask(__name__)

@app.route('/')
def test():
    return 'test'

if __name__ == '__main__':

    app.run()

The debugger display below.
monkey.patch_all() seems to cause it.
Comment out this line from bbc_app.py, warning does not appear but I couldn't understand its comment out is good or bad.

~/bbc1/venv/lib/python3.7/site-packages/bbc1/core/bbc_app.py:19: MonkeyPatchWarning: Monkey-patching outside the main native thread. Some APIs will not be available. Expect a KeyError to be printed at shutdown.
  monkey.patch_all()
~/bbc1/venv/lib/python3.7/site-packages/bbc1/core/bbc_app.py:19: MonkeyPatchWarning: Monkey-patching not on the main thread; threading.main_thread().join() will hang from a greenlet
  monkey.patch_all()

My flask app would be affected this.
I couldn't research sufficiently to solve this, but I thought it could be better to make this issue than not to do.
Thank you,

The tag in the top README.md always shows failing.
Tests in TravisCI seems to fail in setup.

I have installed the latest version of bbc1 using pip.
But from bbc1.core import bbclib does not work and returns an OSError indicating that libbbcsig.dylib does not exist. (I confirmed both Pipenv and venv)

Python 3.7.2 (default, Feb 12 2019, 08:15:36) 
[Clang 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from bbc1.core import bbclib
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/{PATH_TO_PIPENV}/lib/python3.7/site-packages/bbc1/core/bbclib.py", line 33, in <module>
    import bbc1.core.libs.bbclib_utils as bbclib_utils
  File "/{PATH_TO_PIPENV}/lib/python3.7/site-packages/bbc1/core/libs/bbclib_utils.py", line 29, in <module>
    from bbc1.core.libs.bbclib_config import DEFAULT_ID_LEN
  File "/{PATH_TO_PIPENV}/lib/python3.7/site-packages/bbc1/core/libs/bbclib_config.py", line 22, in <module>
    from bbc1.core.libs.bbclib_keypair import KeyType
  File "/{PATH_TO_PIPENV}/lib/python3.7/site-packages/bbc1/core/libs/bbclib_keypair.py", line 31, in <module>
    libbbcsig = cdll.LoadLibrary(os.path.join(directory, "libbbcsig.dylib"))
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py", line 434, in LoadLibrary
    return self._dlltype(name)
  File "/usr/local/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/ctypes/__init__.py", line 356, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/{PATH_TO_PIPENV}/lib/python3.7/site-packages/bbc1/core/libs/libbbcsig.dylib, 6): image not found

How can I install it successfully?

Currently, BBc-1 does not have any function to gather system statistics.

Error log:

pkg_resources.ContextualVersionConflict: (eth-utils 0.7.4 (/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages), Requirement.parse('eth-utils<2.0.0,>=1.0.1'), {'hexbytes', 'eth-abi', 'web3'})

We need to consider the access control to bbc_core, especially for a enterprise use.

In BBC-1, inter-domain collaboration through cross reference mechanism is one of the important features. At this point, BBC-1 does not have cross reference searching function in a inter-domain environment.

This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.

Rate-Limited

These updates are currently rate-limited. Click on a checkbox below to force their creation now.

• Update dependency pytest to v6.2.5
• Update circleci/python Docker tag to v3.10
• Update dependency attrs to v21.4.0
• Update dependency cffi to v1.17.0
• Update dependency py to v1.11.0
• Update dependency pycparser to v2.22
• Update dependency python-dateutil to v2.9.0.post0
• Update dependency zope.event to v4.6
• Update dependency zope.interface to v5.5.2
• Update dependency attrs to v24
• Update dependency chardet to v5
• Update dependency greenlet to v3
• Update dependency iniconfig to v2
• Update dependency mysql-connector-python to v9
• Update dependency packaging to v24
• Update dependency pluggy to v1
• Update dependency pyOpenSSL to v24
• Update dependency pyparsing to v3
• Update dependency pytest to v8
• Update dependency zope.event to v5
• Update dependency zope.interface to v7
• Update ubuntu Docker tag to v24
• 🔐 Create all rate-limited PRs at once 🔐

Edited/Blocked

These updates have been manually edited so Renovate will no longer make changes. To discard all commits and start over, click on a checkbox.

• Update dependency urllib3 to v1.26.19 [SECURITY]
• Update dependency click to v8
• Update dependency itsdangerous to v2

Open

These updates have all been created already. Click a checkbox below to force a retry/rebase of any.

• Update dependency protobuf to v3.18.3 [SECURITY]
• Update dependency Flask to v2 [SECURITY]
• Update dependency Jinja2 to v3 [SECURITY]
• Update dependency Werkzeug to v3 [SECURITY]
• Update dependency certifi to v2023 [SECURITY]
• Update dependency cryptography to v42 [SECURITY]
• Update dependency gevent to v23 [SECURITY]
• Update dependency idna to v3 [SECURITY]
• Update dependency greenlet to v1.1.3.post0
• Update dependency mysql-connector-python to v8.4.0
• Update dependency MarkupSafe to v2
• Click on this checkbox to rebase all open PRs at once

Detected dependencies

• Check this box to trigger a request for Renovate to run again on this repository

the image on page 17 of https://github.com/beyond-blockchain/bbc1/blob/develop/docs/How_BBc1_works_v1.0_ja.pdf
"domain_pnig" should be "domain_ping".

When the new domain is created, it doesn't update config.json

The next minor version up, v0.10, will have the basic features enough to operate various systems.
The next version after v0.10 will be v1.0, with bug fixing, refactoring, package rearrangement and document update over v0.10.

MysqlAdaptor doesn't create database in each domain.

In bbclib_asset.py, asset_file_digest is cut off based on id_length as follows:

self.asset_file_digest = hashlib.sha256(bytes(asset_file)).digest()[:self.id_length]

The digest is not ID, so it must not be cut off.

In unpacking transaction data into transaction object, the mapping info between user_id and the index of BBcSignature list in a transaction is not correctly recovered if the transaction contains BBcReference and BBcWitness objects concurrently.

Some unnecessary entries are included.
For example, "reference_indices" in BBcEvent is not used anywhere.

Even if count argument is over 20, count is set 20.

There isn't attribute for compromised transaction id in KeyType.

The current document sets are very poor. Documents should be added and updated as they are ready. So, a new branch "feature/doc-update" was created. The update in the branch will be merged even without version update.

Currently, there is no function to know the completion about the transaction insert at the core nodes that the app does not connect directly.

For large-scale global domain, we do need a P2P algorithm, Kademlia.
(Currently, a simple full mesh topology is formed in each domain)

In bitcoin system, an address (which corresponds to user_id in bbc1) is created from its public key. In such a case, if a user forgets (or loses) his/her key, the user can never use his asset.
We plan to develop a library to decouple an identifier and a public key and allow user to update key bindings.

In version 1.1, libbbcsig is updated, so it needs to be rebuild by running prepare.sh.

However, if libs/ directory exists in the top directory, prepare.sh script will fail.
Before running prepare.sh, remove libs directory.

$ python file_proof.py get test.txt

fails with the following message:

Traceback (most recent call last):
  File "file_proof.py", line 487, in <module>
    get_file(file=parsed_args.target_file)
  File "file_proof.py", line 263, in get_file
    ret = bbc_app_client.search_asset(asset_group_id, fileinfo["asset_id"])
AttributeError: 'BBcAppClient' object has no attribute 'search_asset'

It is convenient to prepare a consensus functions in the case that an application wants to perform a consensus algorithm.

requirements.txt includes "bbc1". This causes the conflict between the source codes and the installed pip module in the case that we develop a new function in the source codes. Even though a new function is implemented in the source codes, the installed pip module "bbc1" is used.

"bbc1" in requirements.txt is for Dockerfile_pip. One of the possible solutions would be that Dockerfile_pip installs bbc1 by using "pip install bbc1" instead of pip install -r requirements.txt.

Currently, the bbc_core offers a very simple search method to retrieve the transaction with the specified transaction_id. Probably, we need to consider more convenient functions.

Even when 'use_ledger_subsystem' setting in a domain configuration is true, the ledger subsystem module is not activated. It is activated only when '--ledgersubsystem' option is used for bbc_core.py.

I just checked the current structure of bbclib, and found that the transaction could be composed in bson and bson+zlib in addition to the original dedicated binary format. The format of a transaction (bson, bson+zlib or dedicated one) is described OUTSIDE data structure. In fact, it is specified by the first 2 bytes of the output of bbclib, i.e., a transaction data, as a flag. I believe that from the following reason, either one of bson or dedicated format should be eliminated from the core structure of bbclib:

• Signatures do not protect the flag in a transaction currently. This implies that attackers can modify such flags of transactions and then users cannot correctly decode such modified transactions. Hence the system could be compromised.
• The definition bbclib is currently ambiguous. IMHO, it should be a small 'core' module to sign/verify/compose/decompose transactions, and should not be a composer of wire formats.

From the above observations, I would like to propose to define (de)composers of wire format, i.e., bson and bson+zlib, as output plugins of bbc1, and fix the transaction format only with the dedicated one.

Environment

• macOS Sierra
• Python 3.6.3 (with pyenv)

% ./docker-bbc1.sh gitbuild
git: 'archive-all' is not a git command. See 'git --help'.
mv: rename ../bbc1.tar.gz to ./bbc1.tar.gz: No such file or directory
unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /Users/yuki-sh/Projects/bbc1/Dockerfile_git: no such file or directory
rm: bbc1.tar.gz: No such file or directory

https://github.com/beyond-blockchain/bbc1/blob/develop/docker/docker-bbc1.sh#L35