Giter Club home page Giter Club logo

blockchain-analyzer's Introduction

Blockchain Analyzer: Analyzing Hyperledger Fabric Ledger, Transactions

Description

Each blockchain platform, including Hyperledger Fabric, provides a way to record information on blockchain in an immutable manner. In the case of Hyperledger Fabric, information is recorded as a key/value pair. All previous updates to a key are recorded in the ledger of a Hyperledger Fabric peer, but only the latest value of a key can be easily queried (see documentation). This mechanism makes it challenging to perform analysis of updates to a key. Further, if an application groups together certain keys for application-level functions, it is challenging to analyze them together. In order to perform information provenance, it is necessary to analyze all updates to a single key or group of keys that together comprise an information unit.

To address this issue, we have created a project which can be used to analyze ledger data stored within a Hyperledger Fabric peer. This project can also be used to analyze operational data, such as number of blocks and transactions.

Currently, the project includes:

  1. an Elastic beats module (in Go), that ships ledger data from a Hyperledger peer to an Elasticsearch instance.

  2. generic Kibana dashboards, that allow selection of a particular key and visualization updates to it (channel, id, timestamp etc) - similar to Hyperledger Explorer.

  3. scripts to create Hyperledger Fabric network in different configurations, i.e., basic, mulitchannel, and appledemo.

  4. a program to generate test data

  5. a standalone utility to dump Fabric transaction data as JSON file for loading and analyzing in any document store.

  6. scripts to start and stop Elasticsearch and Kibana.

Eventual goal

The eventual goal of this project is to evaluate transactions of any blockchain using any document databases or search engines such as MongoDB, CouchDB, or Elastic stack.

Project background

This project was developed as part of Hyperledger summer internship program.

Getting Started

We recommend that you start with the instructions for basic network.

Once you have it up and running, you can try the multichannel, and appledemo.

If you want to learn more about fabricbeat configuration or connect it to your own blockchain network, click here.

If you want to understand dataflow involving fabricbeat, click here.

License

The Apache 2.0 License applies to the whole project, except for the stack directory and its contents.

Maintainers:

Salman Baset
Balazs Prehoda

blockchain-analyzer's People

Contributors

balazsprehoda avatar dependabot[bot] avatar ryjones avatar salmanbaset avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

blockchain-analyzer's Issues

fabricbeat setup

when I start to run make setup to run fabricbeat, I saw that kind of issue.

$ make setup
mkdir -p vendor/github.com/elastic
cp -R /opt/gopath/src/github.com/elastic/beats vendor/github.com/elastic/
rm -rf vendor/github.com/elastic/beats/.git vendor/github.com/elastic/beats/x-pack
mkdir -p vendor/github.com/magefile
cp -R /opt/gopath/src/github.com/elastic/beats/vendor/github.com/magefile/mage vendor/github.com/magefile
git init
Reinitialized existing Git repository in /opt/gopath/src/github.com/hyperledger/blockchain-analyzer/agent/fabricbeat/.git/
make -f ./vendor/github.com/elastic/beats/libbeat/scripts/Makefile mage -debug ES_BEATS=./vendor/github.com/elastic/beats
make: invalid option -- 'u'
make: invalid option -- 'g'
Usage: make [options] [target] ...
Options:
  -b, -m                      Ignored for compatibility.
  -B, --always-make           Unconditionally make all targets.
  -C DIRECTORY, --directory=DIRECTORY
                              Change to DIRECTORY before doing anything.
  -d                          Print lots of debugging information.
  --debug[=FLAGS]             Print various types of debugging information.
  -e, --environment-overrides
                              Environment variables override makefiles.
  --eval=STRING               Evaluate STRING as a makefile statement.
  -f FILE, --file=FILE, --makefile=FILE
                              Read FILE as a makefile.
  -h, --help                  Print this message and exit.
  -i, --ignore-errors         Ignore errors from recipes.
  -I DIRECTORY, --include-dir=DIRECTORY
                              Search DIRECTORY for included makefiles.
  -j [N], --jobs[=N]          Allow N jobs at once; infinite jobs with no arg.
  -k, --keep-going            Keep going when some targets can't be made.
  -l [N], --load-average[=N], --max-load[=N]
                              Don't start multiple jobs unless load is below N.
  -L, --check-symlink-times   Use the latest mtime between symlinks and target.
  -n, --just-print, --dry-run, --recon
                              Don't actually run any recipe; just print them.
  -o FILE, --old-file=FILE, --assume-old=FILE
                              Consider FILE to be very old and don't remake it.
  -O[TYPE], --output-sync[=TYPE]
                              Synchronize output of parallel jobs by TYPE.
  -p, --print-data-base       Print make's internal database.
  -q, --question              Run no recipe; exit status says if up to date.
  -r, --no-builtin-rules      Disable the built-in implicit rules.
  -R, --no-builtin-variables  Disable the built-in variable settings.
  -s, --silent, --quiet       Don't echo recipes.
  -S, --no-keep-going, --stop
                              Turns off -k.
  -t, --touch                 Touch targets instead of remaking them.
  --trace                     Print tracing information.
  -v, --version               Print the version number of make and exit.
  -w, --print-directory       Print the current directory.
  --no-print-directory        Turn off -w, even if it was turned on implicitly.
  -W FILE, --what-if=FILE, --new-file=FILE, --assume-new=FILE
                              Consider FILE to be infinitely new.
  --warn-undefined-variables  Warn when an undefined variable is referenced.

This program built for x86_64-pc-linux-gnu
Report bugs to <[email protected]>
Makefile:21: recipe for target 'pre-setup' failed
make: *** [pre-setup] Error 2

"go get github.com/hyperledger/fabric-sdk-go" doesn't working

I try to run this project by following README.md.

but when I start to run Beats Agent, I saw that kind of below issue.

go get -u github.com/hyperledger/fabric-sdk-go
go get: warning: modules disabled by GO111MODULE=auto in GOPATH/src;
	ignoring go.mod;
	see 'go help modules'

Does anyone that kind of issue? I guess it is related to golang or absolute path... but I don't know.

Test agent with Fabric network on a cloud provider

e.g.,

  • create an account on IBM or AWS blockchain platform for Fabric.
  • Create a two node network (using free tier), single channel network.
  • Run dummy application against it.
  • Connect the agent to it, and see if it can get the data. Document it.

fabricbeat: make causing issue "undefined: idtools.Identity"

$GOPATH/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/elastic/beats/libbeat/scripts/Makefile
go build -i -ldflags "-X $GOPATH/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/elastic/beats/libbeat/version.buildTime=2019-09-26T21:23:34Z -X $GOPATH/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/elastic/beats/libbeat/version.commit=74ec2c73aedc79dc6349696d342a6afa8064164e"
#github.com/fsouza/go-dockerclient/internal/archive
../../../../../pkg/mod/github.com/fsouza/[email protected]/internal/archive/archive.go:73:19: undefined: idtools.Identity
$GOPATH/src/github.com/blockchain-analyzer/agent/fabricbeat/vendor/github.com/elastic/beats/libbeat/scripts/Makefile:107: recipe for target 'fabricbeat' failed
make: *** [fabricbeat] Error 2

Any help would be highly appreciated.

fabricbeat 'make update'

I try to start make update but this error comes out.

/opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat$ make update
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Installing mage v1.8.0 from vendor dir.
can't load package: package /opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat/vendor/github.com/magefile/mage: import "/opt/gopath/src/github.com/hyperledger-elastic/agent/fabricbeat/vendor/github.com/magefile/mage": cannot import absolute path
vendor/github.com/elastic/beats/dev-tools/make/mage.mk:9: recipe for target 'mage' failed
make: *** [mage] Error 1

Configure Kibana URL.

Hi,

I have been trying to use this project with my existing elasticsearch and kibana setup. Fabric beat is however not reading the kibana configurations. I did try many things but it always always connects to localhost.
2019-12-02T05:02:07.902Z INFO kibana/client.go:117 Kibana url: http://localhost:5601

Did config in fabricbeat.yml as mentioned in the readme and also in many other places still always localhost. Did anyone try with external kibana for this project?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.