Giter Club home page Giter Club logo

ipfs-social-graph's Introduction

Social App

This software is a small social app with federation on top of IPFS. Shared universal timeline with posts from your curated bubble of followees, plus encrypted DMs. There is presently needed a small server component included here which does some IPNS stuff for the federation and provides profile info to get clients started. It also serves the ui as a mobile-friendly web page. Server operators can limit profile operations to those on an approved list, and bad profiles can be explicitly blocked via a block list.

Demo Video: https://www.youtube.com/watch?v=7Mm2_1Xlpz0

Live demo: https://ipfstoy.chws.ca

Older Demo Videos

IPFS Social Graph

An IPFS Social Graph. Create a social Profile, and use the RSA keys to sign new GraphNodes and Profile versions which link to their previous records immutably. Provide a Web UI to crawl this data using js-ipfs. Provide a server component to federate lists of updated profile tip information among network operators using a go-ipfs instance.

Build / run

Prerequisites For Quick Start: Docker compose

Quick start with LetsEncrypt SSL Certs (replace your CIDDAG_WL_PROFILEIDS, CIDDAG_IPNS_FEDERATIONMEMBERS and CIDDAG_TLS_HOSTNAME with correct values, see further below for explanation of what the variables are for):

cd /home/blademccool/IPFS-Social-Graph/
COMPOSE_FILE=docker-compose.base.yml make build
COMPOSE_FILE=docker-compose.base.yml CIDDAG_TLS_HOSTNAME=your.server.hostname CIDDAG_WL_PROFILEIDS="QmNMLj7t8VzxmSs7K3xxxxxxxxxxxxxxxxxxxxxxxxxxxj,QmYxxxxxxxxxxxxxxxxxxxxxxEvVMiD5eZMSvuHBwqVXTG" CIDDAG_IPNS_FEDERATIONMEMBERS="QmZxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxDgL9jPvb4VBh" make up

Quick start with Self Signed SSL Certs for local testing, will allow anyone with the URL to access it (replace your CIDDAG_IPNS_FEDERATIONMEMBERS with correct values):

cd /home/blademccool/IPFS-Social-Graph/
openssl ecparam -genkey -name secp384r1 -out server.key
openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650
mv server.key server.crt compose/tlsdata/
COMPOSE_FILE=docker-compose.base.yml make build
COMPOSE_FILE=docker-compose.base.yml CIDDAG_IPNS_FEDERATIONMEMBERS="QmZxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxDgL9jPvb4VBh" make up
What is CIDDAG_IPNS_FEDERATIONMEMBERS ?

A comma separated list of IPNS names for servers which are publishing lists of profile ids and their best social graph tip node cid info. The server will crawl through the results of all of them and put together a master list of what it thinks is the best (longest) chain of history for a given profileId. Without participating in federation the server will only know about nodes created by its directly connected profile users.

What is CIDDAG_WL_PROFILEIDS ?

A comma separated list of ProfileIds which will be used to limit the users who can post updates to their profile tip information through the server. Any profile listed on this WL, or any profile that is followed by any on the wl, will be allowed to post through the server.

What is CIDDAG_TLS_HOSTNAME ?

It would be a domain name that you control which you can point at the IP where you are running the server at and then LetsEncrypt will be able to do some auto-magic to set your TLS certificates up. See comments in code for more info. IMPORTANT: the crypto.subtle api is not available to remote clients without TLS enabled, which is why this is critical that the web ui be served via TLS.

Manual build notes (if not doing docker-compose method above.)

Prerequisites for manual build, or Golang 1.15.6 or compatible + existing IPFS node to build and run manually.

Below are using example paths for the project and go resource directories. You will need to fix according to your system. Below we will start an IPFS node which will listen for API on port 7767, get dependencies for the golang project, build the ciddag binary from source using golang 1.15.6 and then run it set to provide a ui via http on port 4588, and to communicate with the running IPFS node on its port 7767. Leave the CIDDAG_TLS_HOSTNAME blank for local http ui on the port specified by CIDDAG_HTTP_PORT container environment var (default 4588). Set a value of a hostname that you control otherwise, and it will do automatic "Let's Encrypt" setup for the TLS cert for the domain. It will NOT WORK over https without a valid cert because otherwise the browser crypto key management api is unavailable.

Start IPFS node:

cd /home/blademccool/IPFS-Social-Graph/;
COMPOSE_FILE=docker-compose.base.yml docker-compose up -d ipfs0

Build:

cd /home/blademccool/IPFS-Social-Graph/src;
GOROOT=/home/blademccool/sdk/go1.15.6 GOPATH=/home/blademccool/go /home/blademccool/sdk/go1.15.6/bin/go get -u ./...;
GOROOT=/home/blademccool/sdk/go1.15.6 GOPATH=/home/blademccool/go /home/blademccool/sdk/go1.15.6/bin/go build -i -o ciddag .  

Run:

CIDDAG_HTTP_PORT=4589 CIDDAG_IPFS_API_SERVER=localhost:7767 CIDDAG_IPFS_KEYSTORE_PATH=./compose/ipfs2/keystore CIDDAG_WL_PROFILEIDS="QmQPy3enk6rHvumMT1u2bnNCEr4QoBiqU2EHXaRUVxmw5p,Qmd7Scc5K1B8JLNoMS4cKATQKemAoSaEq7nFc5b3oQ9F3M" ./ciddag 

Symlink for project code in go: If there are issues building, please try creating a symlink for the name the project is using internally in your gopath src dir, and then attempt the build again.

eg: ln -s /path/to/project/src /yourgopath/src/github.com/blademccool/ciddag

Re-Bundling js (example for one of the many used libs, if desired):

cd /home/blademccool/IPFS-Social-Graph/web
browserify peerutil.js -r -s peerutil -o lib/peerutil.bundle.js

State of the code

This is an experimental proof of concept written in a hurry. I welcome any and all improved reorganization and cleanup pull requests that make sense.

Some things that I know are issues:

  • golang test suite is not too comprehensive, and may need some local configuration related tweaks to pass all tests
  • javascript test suite does not exist
  • no limits on data field sizes or validation for their formatting, some should be imposed
  • lots of junk commented out code all over
  • not using the native IPFS DAG/IPLD concept. I suspect the benefits to doing so may include needing less spidering and verifying code for cids as well as possible performance improvements.
  • code can be better organized. some functions are probably in structs or files which are less appropriate than ideal.
  • everything is all in main package right now
  • code is quite coupled making mocking for tests a challenge.

ipfs-social-graph's People

Contributors

blademccool avatar

Stargazers

Faolain avatar Ѵ∑1L avatar

Watchers

James Cloos avatar  avatar

ipfs-social-graph's Issues

Connectivity automation for federated peers

At the moment I do not have a mechanism for automatically obtaining the fully formed multiaddr to reach a IPNS federation peer. We record the IPNS name and use that to look up the output that the other nodes are creating. But we really should also be attempting to directly connect to those nodes with the IPFS instance for pubsub stuff to work right and for IPNS propagation/TTL stuff to work best.

Proposal here likely includes modifying the delegate publish data structure to include a new section for server multiaddr info. List pulling/merging code will need to know about the new format.

JS Test Suite

As a developer I would like the peace of mind of being able to run a test suite for various parts of the codebase and have it assert that everything important is working. Some things I can imagine being automated and mockable, other things I imagine will need ui scripting with something like cypress or jest. I am not down that rabbit hole yet. The things I would like to be testable with reproducible results:

  • create a set of profiles
  • have them make timeline posts
  • follow one another
  • send dms to each other
  • repost each others content
  • construct the timelines and confirm everything expected was pulled and stored correctly in the correct order for each profiles perspective of the graph.
  • ui operations cause creation of the data they should

Do not load the entire timeline into the DOM

as the timeline grows longer the weight of putting the entire thing into the DOM is becoming too heavy. we need to only show the content that should be on the screen. What should be on the screen though?

  • screen has overflow buffer of certain max amount beyond the bottom of screen
  • new content comes in?
    • is it older than the oldest thing that is already hidden off screen? dont load it into DOM
    • is it newer than that? push it into the dom
  • if we did push content into DOM we should remove any excess overflow, anything that is letting us scroll down too far should just be removed from DOM.

Secure Websocket proxy nginx -> IPFS node

I would like to be able to have the browser js-ipfs code directly connect to a server that has a bit more stable and consistent connectivity. I also believe having this working is a prerequisite to getting pubsub to work between go-ipfs and js-ipfs, which I would eventually like to employ to push rapid updates to connected clients.

As part of the server functionality it includes a go-ipfs node that is needed to be running. The js-ipfs code in browser is capable of connecting to go-ipfs nodes via SECURE websocket due to the browser security model. The go-ipfs out of the box does not support TLS websocket. SOooooo we have to set up nginx as a reverse proxy on /wss path and proxy to the /ws path for the IPFS node. There are some guides out there, I will find more details and put here as I get closer to thinking about actually tackling this.

I believe the gist was

  • set up nginx in the docker
  • set up this nginx to get letsencrypt TLS cert
  • reverse proxy /wss to /ws on the IPFS node
  • ipfs node configuration has to enable the ws but adding some special thing in the config, which I can't recall
    • “/ip4/127.0.0.1/tcp/4003/ws" in the swarm list .... or something like that.

some more info:
ipfs/kubo#3907
ipfs/kubo#5251

BL reload on a timer

At the moment we have an extremely rudimentary block listing system, it is a file in the filesystem called bl.json that is a json dict of profileid -> profilecid but it only uses the keys to add whole profile entirely to bl for operating through the server. Any bl'd profile is unable to do anything through the server, period.

This list is presently only read on startup. We would like to put it on a 90 second timer like some other things in the go server code.

Store fully formed graphnode and profile info in indexeddb/localforage

Stop pulling everything from the network every time. if we pulled a cid for a gn or a profile we should be able to fully bake it with all looked up static info and store it in the browser indexed db. This should allow much faster reload of timeline and massively reduce the network burden upon re-opening the ui.

Unfollow and refollow causes display bug in DMs

I believe it is related to not purging the dms when unfollowing, upon refollow they still exist in memory and the refollow crawling operation will end up adding extra copies of the followeees content. Havent reproduced it since I saw it but I believe this is what happened. If so, the solution should be to purge the dms from the followee out of the ordered array they are held in at the time of unfollowing.

JS Bundling?

would like to add js bundling to the build process for the Docker somehow. Not sure if it should be always on or a build option ... as in development i certainly dont want the code i'm working on to be requiring bundling but anything else could be and that might speed up the page load.

IPNS delegate publish TTL adjust

Experiment to see if non-directly-connected nodes can still resolve IPNS without being stuck on a super-stale record as I'd seen in the past. I think we should drop the Lifetime way down to like 10 minutes and set the TTL to something smaller than that, like 2 minutes, break the internode direct connect and see if the updates still propagate over the network.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.