wetrustplatform / blockform Goto Github PK

The nodes connected to Ropsten network are not syncing.

I suppose we have to download the genesis node and do a geth --datadir=/datadrive init ropsten.json, like we did for Rinkeby.. But where to find the json file?

I propose that we hardcode 3 regions per cloud provider for now, 1 for each continent.

Europe, US, Asia

Sometimes, when deleting a node on Azure, the context deadline exceeds.

Try to use a larger timeout value maybe?

2018/12/21 10:35:26 Error while deleting node bf-bi01r47v Future#WaitForCompletion: context has been cancelled: StatusCode=202 -- Original Error: context deadline exceeded

Make sure to destroy the VM security group on delete.

Be careful, the operation will fail if the VM is not fully deleted first.

The order is currently changing when we refresh the page if a node is being deleted.

This is confusing.

Do this at the gorm level.

We're currently using https://gethstore.blob.core.windows.net/builds/geth-linux-amd64-1.8.13-225171a4.tar.gz

Blockform has a basic chain explorer, but it lacks a search field that would allow jumping quickly to a block, transaction or address.

See how it is done on etherscan.

When the application receives the interrupt signal, wait for all goroutines to be complete before exiting.

See https://golang.org/pkg/net/http/#Server.Shutdown
And https://guzalexander.com/2017/05/31/gracefully-exit-server-in-go.html

The created_at column is a timestamp.
We're currently translating it directly to a date format.
It would be better to display relative times like: 8min ago, 3 days ago.
There must be a lib for this.

If a node status is Deployed, we can then do a web3 call to display the syncing progress.

If the node is synced, display the blockNumber instead.

Display the status of the systemd services in the Health tab.

Services are:

• geth
• nginx (reverse proxy)
• faucet (only avail on private nets)
• sshd

Maybe, also allow to pause/restart some of them (except SSH).

Check the blockNumber against etherscan and reboot geth if our node is lagging behind

Private nets are mostly used for development or test purposes.

A minimal block and tx explorer would be nice to have.

It's also nice to have for public networks anyway.

It should be based on the web3 js lib already included in the project. And the provider has to be the blockform geth node.

Using https://github.com/chromedp/chromedp

The package sshcmd is a small utility to execute remote commands on nodes VMs.

It's currently used to reboot nodes on demand. But we plan to use it for more things like updating geth, setting up SSL certificates, or checking the health of the VM.

For this to be possible, we need to return the stdout and stderr output of the executed commands.

Create two buffers, and set sess.Stdout and sess.Stdout, and return the content of the buffers as byte array or string.

Geth nodes are not very reliable. They are regularly out of sync and need reboot.

Having an high availability endpoint would save us the pain of having to code fallback mechanisms in each of our apps.

Ideally, we should do this in 3 separate steps:

• create a node in zone X of cloud provider A
• create another node in zone Y of cloud provider B
• add a load balancer on top of these two nodes

I think it would be a bad idea to have recipes to create these 3 units at the same time in a same cloud provider. It would make the forms more complex, and limit us to use the same cloud provider, which increases the chances of having both nodes down at the same time.

One thing to explore is to do this at the cloudflare level, and then no programming would be required in blockform. But if it's not possible, then it should look like a form to create a load balancer in any supported cloud providers, in blockform.

If we take that route, we should have a sql table to store these load balancers, and expose them on the dashboard.

https://www.digitalocean.com/products/droplets/

Difficult parts:

• get cloudinit to work
• automatically get a domain

When the user is on a HTTPS url, and some nodes are using HTTP, display a warning box explaining that they have to point a domain and call certbot to setup HTTPS.

We currently use 16.04 LTS

HTTPS is needed by users who connect our nodes through web3 from an HTTPS host.

Self signed certificates won't work.

Certbot is not compatible with domains like amazonaws.com, and some cloud providers won't give you an automatic domains anyway (DigitalOcean and GCP).

So I think we should let the users point a domain name to the node, then give them a button that will trigger certbot in the VM.

The API key of a node can be compromised.

Users may need to generate a new API key for the node endpoints.

Use sshcmd and sed to replace the old API key with a new one.

The nginx reverse proxy must be restarted for the change to take effect.

Don't forget to save it in the database.

Something like this:

It has to use a meaningful language and our color palette

User supplies:

• ssh access
• sudo capability

and blockform will set up the node

Implement the CloudProvider interface for google compute engine

https://cloud.google.com/compute/

Only test things that are subject to crash.

Don't go into crazy refactoring in order to write useless tests.

Node actions like 'Reboot' or 'Delete' are currently in a tab, which is not good UX.

Move them in the header, right aligned to the node name, in a nice dropdown menu.

There is already a template for this called node_titlebar.

Use a bootstrap component for this task.

You can also add a greyed 'Pause' action that we will implement later.

In some cases, deleting a security group on AWS will fail because the security group is still seen as attached to the EC2 instance.

Make sure that the EC2 instance is deleted properly, before we delete the security group.

2018/12/21 13:51:14 Error while deleting node bf-402p3jud DependencyViolation: resource sg-0fc9f93189d373a71 has a dependent object
	status code: 400, request id: 041e74c7-70b6-4a2e-8d11-e3dc11640271

geth supports a file called static-nodes.json where we can set the addresses of some other ethereum nodes (they have to be on the same network).

This file can be created during the cloud-init step. See the cloudinit package.

This should be done for mainnet, rinkeby and ropsten.

Read #36 for more details

I've noticed that the first network ID generated by random is always 81.

We may need to seed to get different numbers.

Some people may want to work only with AWS, or only with GCP, etc.

Check if the environment variables are set, and add only the corresponding cloud providers to a map.

Adapt the form template to only display the available cloud providers.

When setting up the SSL certificates, add a step to add a cronjob to renew the certificates

Do some research to find an appropriate db migration strategy compatible with gorm.

We're currently using gorm automigrate but it is sometimes not enough : it's not able to modify a field.

If an error happens during the VM creation or deletion, invoke another callback

We can do a web3 request to check if the node is listening and show this information in the health tab.

See how it is done for eth.syncing and net.peerCount

When something wrong happens in the node, for example, if a public node lags behind etherscan too much, we should notify blockform. There is already a cron that checks this.

For this, create a new database table of events. Events should have a type, and allow for storing more than errors. It should be able to store things like

• this node has been created at
• SSL has been setup on this node at
• this node has been manually rebooted at
• this node has been paused at
• this node has been automatically rebooted because out of sync at
• the API key of this node has been updated at

And it should be able to display this in the activity feed tab of the node.

The VM should be able to add entries in this table by contacting an http endpoint on the blockform side. This endpoint must be authenticated.

Later, we will add Slack/OpsGenie/Mail integration, so these alerts can be broadcasted on these channels. But this is not part of this card.

Use sshcmd to download the latest version of geth and restart the service.

Expose this functionality in the UI.

When connecting public networks like mainnet or rinkeby, geth sometimes gets out of sync.

It happens usually once a month. A restart of the service is enough to fix the situation.

Setup a cron task to check the blockNumber against etherscan. If the difference is more than 200 blocks, restart geth.

There are a few services that the user may want to stop sometimes:

• The faucet
• The proxy
• Geth ?

You can improve the services.go with a new function allowing to pause these services, using systemd stop <service>. And expose this in the UI.

Hi I read your Readme.md and I download the Json for my GCP cloud and in the Set environment variables part. I dont know where I need to put the credentials, and in the readme dont said if that is before or after of run the command go build && ./blockform.

Hi,
Thanks for your amazing project. Can we implement it on openstack or k8s clusters? Do you have a plan to have integration with them?

It can be useful sometimes