desmos-labs / desmos Goto Github PK

Currently there are no PR and issue templates. We should add those as they will be useful to external contributors

Posting posts, replies, likes, etc are frequent activities. These actions should be separated from the key for asset storage as the posting key may be leaving on the web browser, mobile app for easy accessing with low security measure. There is a subkey proposal on Cosmos-SDK which was first implemented during the Berlin HackAtom 2019. This has already been added to the v0.38 milestone.

Borrowing the concepts from Steemit.com

Posting Key permissions
Use your Posting Key to:

1. Publish a post or comment
2. Edit a post or comment
3. Upvote or downvote
4. Resteem content
5. Follow people
6. Mute accounts

Active Key permissions
Use your Active Key to:

1. Transfer tokens
2. Power STEEM up or down
3. SBD conversion
4. Vote for witnesses
5. Place an order on an exchange
6. Certain profile changes
7. Publish a Witness price feed
8. Create a new user

Owner Key permissions
Use your Owner Key to:

1. Reset Owner, Active, and Posting keys
2. Recover your account
3. Decline voting rights

The current demo is generating a created time and expiry time on each session. This can't be verified and not able to make consensus. It has to be changed to block height.

Context

Currently there is no way for a user to query for all posts that have been stored into the chain state, neither for him to query for a subset of those.

Implementation proposal

To fix this issue, the best thing to do would be to implement a query that allows to optionally search through posts using the following data:

• Search using a specific topic or subspace (once #46 are implemented)
• Search using a specific creator

The query should return paginated lists of posts to avoid having much content delivered.

The staking module should be forked and updated the voting power calculation by the changes of Gini coefficient illustrated in the introduction. We need to find out an efficient way to calculate the Gini coefficient of the validators if we keep the voting power change at every block. Otherwise, we can think of activating the bonded tokens in sections like era in Polkadot and epoch in Solana.

https://towardsdatascience.com/gini-coefficient-and-lorenz-curve-f19bb8f46d66

https://towardsdatascience.com/gini-index-vs-information-entropy-7a7e4fed3fcb

https://godoc.org/github.com/shuLhan/go-mining/gain/gini

Context

As Desmos will be a public chain potentially used by many different applications as their backend storage, we might want to allow such applications to store arbitrary data inside posts.

Some examples can be the following.

• BigDipper might want to store a reference to the chain-id for which the message has been posted and/or the address of the original poster;
• applications having a local storage might want to store a locally-generated reference to a post in order to easily update its status when its uploaded to the chain.

Arbitrary data support proposal

In order to solve both issues in one single run, I suggest we implement the support for arbitrary data by adding a new field into the Post type.

For sake of simplicity I recommend we only support String -> String associations, and not any king of value type that might be used inappropriately.

With this, any developer can integrate whatever data he wants into the post itself, without having to add any overhead to the overall post structure.

Storage implementation

In order to implement such feature, we can change how Post objects are made, adding a new map[string]string field:

type Post struct {
    PostID            PostID            `json:"id"`
    ParentID          PostID            `json:"parent_id"`
    Message           string            `json:"message"`
    Created           sdk.Int           `json:"created"`
    LastEdited        sdk.Int           `json:"last_edited"`
    AllowsComments    bool              `json:"allows_comments"`
    ExternalReference string            `json:"external_reference"`
    Owner             sdk.AccAddress    `json:"owner"`

    OptionalData      map[string]string `json:"optional_data"` // Contains any kind of arbitrary data
}

I also think that we might want to move the existing ExternalReference field into such map to make the overall object more simple and having less fields leading thus to a smaller size on disk:

type Post struct {
    PostID            PostID            `json:"id"`
    ParentID          PostID            `json:"parent_id"`
    Message           string            `json:"message"`
    Created           sdk.Int           `json:"created"`
    LastEdited        sdk.Int           `json:"last_edited"`
    AllowsComments    bool              `json:"allows_comments"`
    Owner             sdk.AccAddress    `json:"owner"`

    OptionalData      map[string]string `json:"optional_data"` // Contains any kind of arbitrary data
}

This is the first time I got this error when I run a CLI operation via a remote node. All operations give me this error.

ERROR: invalid character 'C' looking for beginning of value

Here is the stack trace.

ERROR: invalid character 'C' looking for beginning of value
error unmarshalling rpc response
github.com/tendermint/tendermint/rpc/lib/client.unmarshalResponseBytes
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/rpc/lib/client/http_client.go:362
github.com/tendermint/tendermint/rpc/lib/client.(*JSONRPCClient).Call
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/rpc/lib/client/http_client.go:208
github.com/tendermint/tendermint/rpc/client.(*baseRPCClient).ABCIQueryWithOptions
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/rpc/client/httpclient.go:175
github.com/cosmos/cosmos-sdk/client/context.CLIContext.query
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/client/context/query.go:91
github.com/cosmos/cosmos-sdk/client/context.CLIContext.QueryWithData
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/client/context/query.go:42
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.GetAccountWithHeight
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:45
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.GetAccount
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:32
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.EnsureExists
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:60
github.com/cosmos/cosmos-sdk/x/auth/client/utils.PrepareTxBuilder
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:304
github.com/cosmos/cosmos-sdk/x/auth/client/utils.CompleteAndBroadcastTxCLI
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:49
github.com/cosmos/cosmos-sdk/x/auth/client/utils.GenerateOrBroadcastMsgs
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:40
github.com/cosmos/cosmos-sdk/x/bank/client/cli.SendTxCmd.func1
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/bank/client/cli/tx.go:53
github.com/spf13/cobra.(*Command).execute
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/tendermint/tendermint/libs/cli.Executor.Execute
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/libs/cli/setup.go:89
main.main
	/Users/creativeworks/Projects/desmos/cmd/desmoscli/main.go:64
runtime.main
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/asm_amd64.s:1357
ABCIQuery
github.com/tendermint/tendermint/rpc/client.(*baseRPCClient).ABCIQueryWithOptions
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/rpc/client/httpclient.go:179
github.com/cosmos/cosmos-sdk/client/context.CLIContext.query
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/client/context/query.go:91
github.com/cosmos/cosmos-sdk/client/context.CLIContext.QueryWithData
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/client/context/query.go:42
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.GetAccountWithHeight
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:45
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.GetAccount
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:32
github.com/cosmos/cosmos-sdk/x/auth/types.AccountRetriever.EnsureExists
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/types/account_retriever.go:60
github.com/cosmos/cosmos-sdk/x/auth/client/utils.PrepareTxBuilder
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:304
github.com/cosmos/cosmos-sdk/x/auth/client/utils.CompleteAndBroadcastTxCLI
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:49
github.com/cosmos/cosmos-sdk/x/auth/client/utils.GenerateOrBroadcastMsgs
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/auth/client/utils/tx.go:40
github.com/cosmos/cosmos-sdk/x/bank/client/cli.SendTxCmd.func1
	/Users/creativeworks/go/pkg/mod/github.com/cosmos/[email protected]/x/bank/client/cli/tx.go:53
github.com/spf13/cobra.(*Command).execute
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:826
github.com/spf13/cobra.(*Command).ExecuteC
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:914
github.com/spf13/cobra.(*Command).Execute
	/Users/creativeworks/go/pkg/mod/github.com/spf13/[email protected]/command.go:864
github.com/tendermint/tendermint/libs/cli.Executor.Execute
	/Users/creativeworks/go/pkg/mod/github.com/tendermint/[email protected]/libs/cli/setup.go:89
main.main
	/Users/creativeworks/Projects/desmos/cmd/desmoscli/main.go:64
runtime.main
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/proc.go:203
runtime.goexit
	/usr/local/Cellar/go/1.13.4/libexec/src/runtime/asm_amd64.s:1357

The closest issue I found on SDK GH is this one. cosmos/cosmos-sdk#3900

Currently there is an entire folder of JSON files (examples) that are not used inside the application nor its tests. I think that this folder should be deleted as well as the file unsigned-cs.json inside the root folder.

Context

Ideally, what we want to achieve with Desmos is to enable any developer to create a messaging-based application that is completely decentralized and censorship resistant.

To do this, we need to find a way to distinguish the messages of an application from the ones of other applications built on Desmos.

Subspace implementation proposal

To solve this problem, we can implement the concept of subspace and topic.

A subspace will identify all the messages coming from the same application. For example, all messages coming from BigDipper will have subspace = "bigdipper".

A topic, on the other hand, allows applications to arbitrarily decide the topic of a specific post. For example, inside the bigdipper namespace we might want to have proposal-discussion, private-message and other topics that let the clients easily fetch the messages related to a specific discussion subject.

Storage implementation

The post object should be changed adding two new fields:

type Post struct {
    PostID     PostID         `json:"id"`
    ParentID   PostID         `json:"parent_id"`
    Message    string         `json:"message"`
    Created    int64          `json:"created"`
    LastEdited int64          `json:"last_edited"`
    Owner      sdk.AccAddress `json:"owner"`

    Subspace   string         `json:"subspace"`
    Topic      string         `json:"topic"`
}

The following data should also be stored:

• The associated list of post IDs that are part of a specific subspace.
• The list of topics associated to a specific subspace
• The list of post IDs that are part of a specific topic.

In this way we will be able to fetch in constant time the following data:

• The list of messages for a given subspace
• The list of topics for a given subspace
• The list of messages for a given topic
• The subspace of a post
• The topic of a post

A chain should be registered on Desmos in order to use if messages are sending from other chains.

Context

As a social network enabling blockchain, one core feature that we should have is the possibility for users to create their very own decentralized profile.

In the current era of social networks, profile are most of the time made of the following user data:

• Name
• Surname
• Moniker
• Mobile phone
• E-mail address
• Address of residence
• Date of birth
• Sex

Also, a part from these privacy-related stuff, people can often also set

• A profile picture
• A cover picture
• A bio/description

Personally, I would avoid putting any privacy-related information on-chain as this can lead to leak of data and spam/scam actions.

For this reason, I propose to create a profile that contains only the following information:

On the other hand, I would exclude the following data:

• Mobile phone
• E-mail address
• Address of residence
• Date of birth
• Sex

Also, as other social networks already exist, I propose to allow the users to connect such socials to our profile, saving a list of social network connections.

Implementation proposal

Based on the considerations written above, the implementation I propose is defined by the following structures:

// Account represents a generic account on Desmos, containing the information of a single user
type Account struct {
	Name     string       `json:"name,omitempty"`
	Surname  string       `json:"surname,omitempty"`
	Moniker string       `json:"moniker,omitempty"`
	Bio      string       `json:"bio,omitempty"`
	Pictures *Pictures    `json:"pictures,omitempty"`
	Links    []SocialLink `json:"links,omitempty"`
}

// Pictures contains the data of a user profile's related pictures
type Pictures struct {
	Profile string `json:"profile,omitempty"`
	Cover   string `json:"cover,omitempty"`
}

// SocialLink represents a single link to another social network
type SocialLink struct {
	Social string `json:"social"`
	Uri    string `json:"uri"`
}

Social links constraints

As we live in a world where catfishing and identity stealing is increasing over time, I suggest we adopt a series of precautions about other social network connections.

Restricting accepted social networks

I would suggest we create a system inside which we do not accept all social networks to be linked to Desmos but, instead, only a subset of the existing ones. This is due to prevent hardly-verifiable social networks to be linked, which would cause possible identities steals.

The list should be done on-chain and managed thought an on-chain governance. The following actions should be supported:

1. Adding a new social network
2. Removing an existing social network

Specifying the authenticity controls

For each future supported social network, there should be a document defining properly how the authenticity of a connection to such social network can be proven. This document should be approved by the overall community before the social network itself is marked as fully supported on Desmos.

Context

During the same conversation already mentioned inside #60 that I had with @angelorc yesterday, another possible problem that he raised about our chain is regarding the storage of posts.

He told me that in Bitsong they have been testing through all the year different methods of storing songs data and metadata so that the chain can always perform good without becoming larger than it should be. He told me that the best way they found out in the end was storing all the metadata on IPFS and linking the CID (the IPFS hash of the file containing such data) on the chain itself.

After such conversation, I've done some hypotheses on how large our chain could become if we keep all the posts contents on-chain. This is what I got.

Chain space analysis

Premises

All the following analysis have been done referring to the JSON format of posts, which is the one used when exporting them inside the genesis or importing them during an upgrade process. This has been done due to the fact that is simpler to verify how large a JSON file is instead of how much space it occupies inside KV stores.

Current status

Currently Posts are stored using the following JSON structure:

{
  "id": "134",
  "parent_id": "156",
  "message": "This is an estimated average long post message",
  "created": "566879",
  "last_edited": "899978225",
  "allows_comments": false,
  "subspace": "dwitter",
  "optional_data": {
    "local_id": "e3b98a4da31a127d4bde6e43033f66ba274cab0eb7eb1c70ec41402bf6273dd8"
  },
  "owner": "desmos1wjtg20d7hl9y409hhfydeqaph5pnfmzxlgxjg0"
}

In this case I've put an average length value for each field, considering the optional_data value to have only one key inside, which should be the average case. This JSON has a size on disk of 378 bytes.

Now, with the introduction of media-supporting posts (#36), an average JSON file of such time could look like the following:

{
  "id": "134",
  "parent_id": "156",
  "message": "This is an estimated average long post message",
  "created": "566879",
  "last_edited": "899978225",
  "allows_comments": false,
  "subspace": "dwitter",
  "optional_data": {
    "local_id": "e3b98a4da31a127d4bde6e43033f66ba274cab0eb7eb1c70ec41402bf6273dd8"
  },
  "owner": "desmos1wjtg20d7hl9y409hhfydeqaph5pnfmzxlgxjg0",
  "medias": [
    {
      "uri": "http://ipfs.pics/QmXr1iejP2zHptFFDr3hycZvbaXaQNwrK6VVXYbxFAYQ7x",
      "mime_type": "image/png"
    },
    {
      "uri": "http://ipfs.pics/QmTbx9HLaN7gsKiunNc5NWvNRytydKn5uWWmpUzAHPU3NS",
      "mime_type": "image/png"
    },
    {
      "uri": "http://ipfs.pics/QmQZWZxfSHB3FpU8w4EMfX9bF52wUyPCiVNjtjf6jeNBBp",
      "mime_type": "image/png"
    }
  ]
}

Due to the higher amount of information stored, such JSON has an on-disk size of 768 bytes.

Let's now consider the case in which Desmos gains 1/1000th of Twitter usage. This could lead to 2 millions posts per each (source of Twitter usage stats: InternetLiveStats).

Considering an average of 1/5 posts being a media post, this would lead to 400.000 media posts/year and 1.600.000 text posts/year created. Considering 378 bytes per each text post and 768 bytes per each media post, this would sum up to approximately 0.92GB of posts per year.

If we scale up to 1/100th of Twitter usage, we then would collect 9.2GB/year of only posts. This amount is crazily high and I personally think that we should reduce it as in the long run it might become a problem considering that there are a lot of posts type we still need to define (e.g. #14).

Solution

The solution of storing all the post content into an IPFS file could be a good way to decrease the size of the chain on-disk. What we could do is define a new Post structure that is stored like the following:

{
  "id": "87556",
  "created": "566879",
  "content": "QmP8jTG1m9GSDJLCbeWhVSVgEzCPPwXRdCRuJtQ5Tz9Kc9",
  "owner": "desmos1wjtg20d7hl9y409hhfydeqaph5pnfmzxlgxjg0"
}

Inside the IPFS file reachable using the content reference, we can then have different JSON structures based on the type of the post itself. As an example, we could have

Text post

{
  "type": "post/Text",
  "parent_id": "156",
  "message": "This is an estimated average long post message",
  "allows_comments": false,
  "subspace": "dwitter",
  "optional_data": {
    "local_id": "e3b98a4da31a127d4bde6e43033f66ba274cab0eb7eb1c70ec41402bf6273dd8"
  }
}

Media post

{
  "type": "post/Media",
  "parent_id": "156",
  "message": "This is an estimated average long post message",
  "allows_comments": false,
  "subspace": "dwitter",
  "optional_data": {
    "local_id": "e3b98a4da31a127d4bde6e43033f66ba274cab0eb7eb1c70ec41402bf6273dd8"
  },
  "medias": [
    {
      "uri": "http://ipfs.pics/QmXr1iejP2zHptFFDr3hycZvbaXaQNwrK6VVXYbxFAYQ7x",
      "mime_type": "image/png"
    },
    {
      "uri": "http://ipfs.pics/QmTbx9HLaN7gsKiunNc5NWvNRytydKn5uWWmpUzAHPU3NS",
      "mime_type": "image/png"
    },
    {
      "uri": "http://ipfs.pics/QmQZWZxfSHB3FpU8w4EMfX9bF52wUyPCiVNjtjf6jeNBBp",
      "mime_type": "image/png"
    }
  ]
}

Of course this are just generic examples and better schemes should be defined more in depth and maybe event stored on chain for a generic reference.

On-disk space changes

Thanks to this approach, all different posts would have an on-disk size of just 166 bytes, which would reduce the above yearly disk-space increase from 0.92GB/year to 0.332GB/year (−69,456%) considering 2.000.000 posts/year.

The best thing is that this approach keeps a linear increase of the disk space that is not based on the posts contents at all. If all users suddenly started creating media posts, on chain they would weight the same as text posts so it's also a spam prevention system.

Querying

Thank to the system proposed inside #60, the parser could simply ready the posts from the chain and then get the real content from IPFS, index it and putting into the database allowing for easier queries.

Consideration on IPFS

IPFS has proven to be reliable and we can even improve its availability by creating a cluster or a private network so that posts content will be always available, although I don't think this will be needed.

Cons of this

The only con of this is that the chain would not be able to check the contents of the posts during the transaction processing. However, I don't know if that might be a problem for us or not.

Also, clients would have to upload the content before the transaction, so we are moving onto them this responsibility. This can however be fixed by creating a REST API that performs such operation for the client.

Conclusion

I personally think this should be the way to follow that would allow us to:

• focus less on the development of the posts module.
  Once you define the specification for each post, you do not need to edit the way they are stored on chain.

• achieve a higher scalability of the chain itself.
  As posts are lighter on disk, the chain can scale better and transactions will be processed faster.

I would love to hear @kwunyeung and @bragaz feedback on this and what it might be improved or should change on such approach.

We should let other platforms to run "smart contracts" (I always don't like this term tbh) using already wide-adopted languages like JS and Python. There are a few Go specific VM for the both languages. As long as Desmos stick to the limited available features for social network, the overhead of running those scripts should not be too heavy. If really too heavy, we can introduce off-loading some tasks to GPU.

Context

In the era of current social networks, more and more applications are allowing users to add more than simple likes to posts. As an example, we've seen Facebook reactions:

Another one is the ability on Slack to react with emojis:

I think we should support this feature too as it make Desmos even more attractive to applications developers.

Post reaction implementation proposal

Currently we represent the like object using the following data structure:

type Like struct {
    Created sdk.Int        `json:"created"` // Block height at which the like was created
    Owner   sdk.AccAddress `json:"owner"`
}

I propose to expand and rename this structure to support other reaction types as follows:

type Reaction struct {
    Created sdk.Int        `json:"created"`
    Owner   sdk.AccAddress `json:"owner"`
    Value   string         `json:"value"`
}

The only change is the new Value field which allows application to insert any kind of text inside the reaction. This might be a simple like string, or even emojis.

Storage changes

No storage changes should be required.

I can't actually push anything to github from my IDE even if i set up my SSH key properly.

Remote: Permission to desmos-labs/desmos.git denied to bragaz. unable to access 
'https://github.com/desmos-labs/desmos.git/': The requested URL returned error: 403

Currently there are a bunch of methods inside keeper.go that have double return values.
As examples, we can take a look at

I personally think that having a double return value with a bool value that is true when there was no error is useless. It's probably better and more efficient to have a single error value which is nil if everything went ok and then simply having to check that value without having to do the double err != nil && success check.

A user can flag inappropriate of messages. Conditions of hiding messages should be implemented. For example, the message will be hidden if at least be flagged by 2 individuals.

The user of the flagged message can appeal. The appeal should be juried by the community, maybe via some kind of governance?

Poll is very common in communications. A Poll will have poll options and aggregated results.

Actions

• Migrate the currently used Cosmos SDK to v0.38.0
• Write the documentation on how to migrate to such version, as seen from
  • Users
  • Validators

Is your feature request related to a problem? Please describe.
Currently with PR #37 we've switched from using time.Time values to int64 when representing the duration time of magpie sessions.

Taken from @kwunyeung's #37 (comment) we should think about having a parametric session duration time.

Describe the solution you'd like
I suggest leaving the user the ability to specify a session length (in blocks) when creating it. This could be done by adding a new duration parameter to the MsgCreateSession type.

I would also like to know more about what @kwunyeung meant with "We should add this to the Genesis params later" in order to know if we can implement such thing.

There are so many dependencies in the Cosmos SDK which uses Cobra and Viper. These dependencies make the desmoscli can't be compiled purely to wasm. We should figure out a way to remove those dependencies to build a standalone wasm binary which can be run on browser and mobile app directly.

Context

Right now the app folder has a test coverage of 5%. I think it's time we implement some simulations test like Cosmos did, bringing up such value and avoiding future problems that might arise during migrations and/or chain starts.

NOTE. Once implemented, the CI should be changed too to run simulations. Cosmos has a good reference

There is a TensorFlow-Go implementation which can run Python TensorFlow codes in Go application. It does worth to take a look and see if this can be done in a distributed way to provide better user engagements in some certain ways.

https://www.tensorflow.org/install/lang_go

Currently when starting the chain, after the genesis creation, we need to remember to edit the genesis.json file and change the bond_denom param inside the staking module from stake to desmos.

I personally find this thing a little bit tedious to do by hand and I think we can automate it by creating a wrapper staking module that acts exactly as the default one but simply sets the proper bond_denom value (desmos) during genesis.

This would allow for easier on-boarding of new developers and faster local development of the chain itself.

Context

Yesterday, during a talk with @angelorc from Bitsong, we were confronting each other on chain state approaches that would allow our chains to run smoothly when the following question arose:

Why are we creating REST queries endpoints on full nodes?

To be honest, I really don't get it too.

Currently we've built them so that clients can fetch data without having to look at blocks or transactions directly. In the future, however, this might become a problem if the set of data becomes so large that queries can potentially block a node from operating normally.

The main problem of the on-node queries approach is that nodes do not perform index-based queries. Instead, they iterate over the whole set of data and filter out the objects that do not satisfy a given filter. This is highly inefficient and can lead to a lot of problems if the data to filter is really large and the filtering process takes a lot of time.

As a comparison:

• on Twitter users create more than 200 billion tweets per year (Source);

• on Facebook every 60 seconds: 510,000 comments are posted, 293,000 statuses are updated, and 136,000 photos are uploaded (Source)

Assuming to have even 1/100th of just Twitter traffic, this would mean that we would have to handle 2 billion posts per year. By the end of the first year, assuming 1 microsecond to iterate over 1 post, filtering through all the posts could then take up to 2000 seconds, which are 33 minutes!

Middle layer storage and querier

In order to solve this problem, Angelo came up with a really good idea that in my opinion is the one we should adopt too, helping him realizing it.

The solution is pretty simple:

Delete all the module-specific query endpoints from the nodes and create a middle layer data storage and querier application based on real databases

The particularity in this approach lies on the fact that we will use the blockchain only as the state and not the database. Instead, the database will be created on top of the state by parsing the transactions and reading the data inside them.

Here is a schema of a possible system:

As you can see, the middle layer is divided into the parser, the database and the APIs parts.

Parser

It's job should be to read the genesis of the chain and any consequent transaction that is performed inside it. It should fetch such data and store them into a database that allows for easier queries.

Database

The database should just hold data inside an indexes-based system. In this case, the best solution available AFAIK is a Mongo replica set which allows for easy clustering and high fault tolerance.

APIs

The APIs should provide easy data access to clients that request them. Two possible solutions can be provided: GraphQL, with real-time endpoints that updates once a new data is inserted inside the chain, and REST APIs**, that allow for easy and broadly usable data queries.

Decentralization

Although this might not seem the best way to decentralize things, this whole system should not be the only one that acts as described. However, it should instead be just a reference implementation of how things should be done for people wanting to base their application on Desmos, and should be 100% open source so that new projects can easily fork any part of it and create their own middle layer application the best way they want.

Conclusion

I personally think this system should be the way to go for our use case, allowing clients to read data easier that they do right now and in a most efficient way. However, I'd like to gather any feedback on such system and what do you think could be changed or improved about it @kwunyeung @bragaz

The current Cosmos SDK expose a RESTful API. Although we target to have a fully standalone WASM client, we should keep the LCD for other use cases. The LCD is a RESTful API with a lot of specific endpoints. We can study if there are possibilities to change this to be GraphQL which can reduce the number of queries when using RESTful API. If this works, we can contribute back to the SDK.

https://github.com/graphql-go/graphql

Oh suddenly I found that I was wrong. I thought it would be cool to have the block height as time reference but the block time won't be available if the blocks will be clear out in chain upgrade. Should revert it back to timestamp so that we can get it directly from the chain state. My bad.

Context

With #52 we introduced the ability to put arbitrary data inside each post. This is particularly useful to developers when wanting to attach any external data that might be later helpful to a post.

However, while highly useful, it also might become a problem if some attacker starts using this field to put crazily long data inside it to increase the chain storage enormously.

Solution

What I have thought is to limit the number of arbitrary data that can be stored per post as well as the length of each data stored. The limitations I have thought are:

• maximum 10 fields per post
• maximum 200 chars per field

This will lead to at most 2000 bytes-length arbitrary data fields, which IMO will be sufficient for most developers as well as reasonably low to store on chain.

In the future, if needed, we can increase both limits independently.

What do you guys think? @kwunyeung @bragaz

Currently there is no CI/CD setup into the repo. We should use CircleCI as well as add proper unit tests

Set CoinType to 852

Describe the bug
Currently when storing the like of a user to a post, we do not check if the user has already liked the post itself. This leads to the possibility of double-liking.

To Reproduce
Steps to reproduce the behavior:

1. Create a post
2. Like the post
3. Like the post again

Expected behavior
An error should be returned when trying to like the post twice.

Add script to Makefile to generate testnet automatically. The script will do the follow steps.

1. Init genesis file
2. Update bond denom
3. Create 4 .desmosd and .desmoscli home directories
4. Create 1 key in each .desmoscli home directory
5. Add genesis accounts to all 4 addresses
6. Create gentx with the 4 keys from different directories
7. Copy all gentx to one single gentx directory
8. Collect gentxs
9. Copy the generated genesis file to each .desmosd/config directory
10. Update ports to the genesis files in each .desmosd/config directory
11. Run make install
12. Start 4 nodes with 4 different .desmosd home directories

I have created a wasm branch to try to build some query functions in wasm but failed. Here are the compile errors.

$ GOOS=js GOARCH=wasm go build -tags="nofuse purego" -o desmos.wasm ./wasm/desmosw
# github.com/bgentry/speakeasy
../../go/pkg/mod/github.com/bgentry/[email protected]/speakeasy.go:23:18: undefined: getPassword
# github.com/go-kit/kit/log/term
../../go/pkg/mod/github.com/go-kit/[email protected]/log/term/term.go:14:6: undefined: IsTerminal
# github.com/spf13/viper
../../go/pkg/mod/github.com/spf13/[email protected]/viper.go:286:19: undefined: fsnotify.NewWatcher
../../go/pkg/mod/github.com/spf13/[email protected]/viper.go:307:20: select case must be receive, send or assign recv
../../go/pkg/mod/github.com/spf13/[email protected]/viper.go:334:18: select case must be receive, send or assign recv
# github.com/syndtr/goleveldb/leveldb/storage
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:107:16: undefined: newFileLock
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:192:3: undefined: rename
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:267:12: undefined: rename
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:272:12: undefined: syncDir
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:555:9: undefined: rename
../../go/pkg/mod/github.com/syndtr/[email protected]/leveldb/storage/file_storage.go:591:13: undefined: syncDir

I believe there are some OS specific dependency which it cannot find those function when the GOARCH is targeted to wasm.

Currently there is no documentation at all about message types, starting a new chain etc.
We should definitely create a docs.desmos.network website where developers can get an insight on how to build a Desmos-based application.

Sometimes I would like to Like an activity like this on Big Dipper subconsciously.

Desmos should have an ability to Like such activities as a link and incentivize those being Liked. In this case, if I liked the activity of B-Harvest, B-Harvest will be incentivized and the record will be linked to their Desmos address.

With fixes and NFT

The post with ID 44 was edited before. After the edit, children list of the parent post with ID 42 has two 44 there. It should be only one.

http://lcd.morpheus.desmos.network:1317/posts/42

{
height: "0",
result: {
id: "42",
parent_id: "0",
message: "DEStroy MOdern Slavery in 2020",
created: "286474",
last_edited: "0",
allows_comments: true,
external_reference: "",
owner: "desmos1jzju74lns6ldhfs9zqt256uuvh0fk98xlzlzaw",
likes: [
{
created: "286699",
owner: "desmos1fc3mdf0ue2f4suyg5vjj75jtaer0cl0dgqvy6u"
}
],
children: [
"43",
"44",
"44"
]
}
}

Should we record all changes like FB?

This will be very useful for proposal discussions on Cosmos Hub

Is your feature request related to a problem? Please describe.
Currently when fetching posts using the REST API there is no way to sort the posts before the pagination. This results into the impossibility of fetching (e.g.) the latest posts sorted by descending creation date.

This is particularly useful when developing applications that needs to show the posts in a timeline.

Describe the solution you'd like
Improve the /posts REST API endpoint to accept a sorting parameter and a sorting order parameter. One example could be

/posts?sort=created&sort_order=descending

Describe alternatives you've considered

Additional context

Describe the bug
The /posts REST endpoint does not behave as expected

To Reproduce
Try fetching some posts using /posts?page=1&limit=10. No posts will be fetched.

Expected behavior
Posts should be fetched properly.

Screenshots
N.A.

Additional context
N.A.

subspace in Post is better in hash value so that it can be verified by the users by comparing the hash value instead of relying on comparing with a string.

Consider an imposter is trying to build an app and put subspace as C0SMOS to pretend to be an app discussing things on COSMOS.

But I don't have an idea of how the UX can be done to let normal endusers to verify this yet. @RiccardoM @bragaz please comment.

These txs are all failed and have error message with the same Post ID 17.

828D4A5EE5488F1B1C67CC72410B2F62CEF7A67DFB4F6EF2CE1FA550136DCDF0
4951F960C4A8D222175D377138D1B9519A79FC609E6410B03C71361067FF4BEC
6FB74064F5038995FB1DF9FFA246C96B1AF5A5D3DD4A1F452DC5AA299C877809
CAEB1E82282EE31F5CAE3D3E99838DBD62064142C90C480CF29976917A88742E
D262923277A6FD586A40289E3BFB7D5067FAE1335A9B6652B1CF5B8A8A77C9C2

Their error messages are all

"{\"codespace\":\"sdk\",\"code\":6,\"message\":\"Post with id 17 already exists\"}"

Currently all the features are present inside the magpie module. I personally think that we should split those features into different modules. They might be the following ones:

• magpie for cross-chain sessions
• posts for posts and likes
• discussions for discussions proposed into #13
• polls for polls proposed into #14

Context

As discussed inside #61, we should limit the length of posts to 500 characters. Longer should be uploaded to IPFS in order to avoid an uncontrolled grow of the chain on-disk size.

Solution

The solution is pretty straightforward and can be implemented as a check inside the MsgCreatePost, MsgEditPost and Post#Validate methods:

if len(post.Message) > 500 {
  // return error
}

Propose adding a Media and MediaType field in Post module for storing a media file in a post. Similar to an image of a Twitter tweet or an image post of Instagram.

This field is optional.
If it exists, validate if the Media value is a valid url. MediaType should be an option from a list of mime types.

Currently the GetX methods inside the keeper.go file have a single return value. As examples we can take a look at

I personally think that in this case having a single value may make the "not found" situation hard to handle.

Let's consider the first case (GetPost). As this returns a default Post when none is found, what should we check to make sure that is was not found indeed?

I think that having a double return value such as the following might be more useful:

func (k Keeper) GetPost(ctx sdk.Context, id string) (post Post, found bool) {
    // ...
} 

Having a method like this one, we could then perform a simple check like this one

if post, found := keeper.GetPost(ctx, id); found {
  // do something with post 
}

Or something like this

post, found := keeper.GetPost(ctx, id) 
if !found {
  // throw an error 
}

// do something with post

desmosd panic with this during InitGenesis.

panic: unsupported type map

It looks like not the issue of the message is too long and can't be saved. I have reduced post id 46 to less than 500 characters but still can't save the states.

The issue should be the InitGenesis passing a wrong object to SavePost and it gets panic when it gets a map instead of a Post during MustMarshalBinaryBare due to the change of the genesis file structure before for including reactions.

One of the features in our proposed tokenomics is to let all validators have the same chance of proposing block so that the proposer bonus will and rewards will be distributed in in the same chance. The idea is to break the relationship between number of delegated tokens and proposing priority.

To achieve this, there are two approaches.

1. We set voting power of all validators in the active set to be equal. This is how IOV works. And override how the delegation in the Cosmos SDK which it will not update the voting power based on the delegated tokens. In this approach, each proposer will have equal weight of chance in proposing blocks based on the current proposer priority mechanism.

2. A long term solution is to apply the random selection of block proposer by using Verifiable Random Function. This is current currently Algorand is using for selecting consensus committee. The approach has also been discussed in Tendermint.

Some validators or delegators on different chain may be using a multisig address to operate, e.g. Sikka and Iqlusion. The current create session function only validate with an account with a single signature. The create session function should support multisig addresses.

I might be completely wrong, but I think that currently inside the handleMsgCreatePost there is a non deterministic operation, which is

post := types.Post{
    ID:            xid.New().String(),
}

Seeing the code inside the xid.New() method, it uses the current time to generate a globally unique identifier. If this code runs on multiple nodes, it won't produce the same effect.

I personally think we have two options:

1. Find a way to change how this id is generated, making it deterministic
2. Allow clients to specify it and just check for its validity (using a regex) and uniqueness

I think the second option is the best one, and would also mean that we can simplify the MsgCreatePost by asking the client to send a full Post object and later changing its likes number to 0.