In order to help people find things we need to more narrowly define some of our repos. Some repos are very broad, overlap with others, or aren't defined at all. We also need to update the descriptions to reflect our definitions.
Some ideas:

• ipfs/infrastructure
  What I propose: Scripts and issues for any infrastructure that isn't a standalone repo
  Currently defined as: Infrastructure for IPFS users + devs
• ipfs/community
  As it stands this serves as infrastructure documentation and news for the ipfs community. We may
  want to split this up into a repo for infrastructure documentation and a repo for news. Since people
  are already subscribed to ipfs/ipfs we may want that to be our news channel. We could then
  rename this to ipfs/infrastructure-docs. I would even be a fan of removing this repo altogether and
  throwing it all into ipfs/infrastructure. For instance,
  https://github.com/ipfs/community/blob/master/dev/tools/hooks/commit-msg seems like it fits into
  the definition of ipfs/infrastructure
  Currently defined as: IPFS community
• ipfs/ipfs
  This repo has a large number of people "watching" it. That means they get notified on new issues.
  We need to use this to our advantage and not scare these people away. I believe the original
  purpose was to keep people notified of progress on ipfs. Using this as news would be good.
  Currently defined as: IPFS - The Permanent Web http://ipfs.io

Have documentation on how the kanban board at https://waffle.io/ipfs/ipfs works.
See #7 (comment)

Notes:

• Github is picky about the syntax around "fixes #num" when linking a pull request to an issue
• Pull requests that fix an issue and are themselves "in progress" should move that issue into "in progress." (appears to simply be a bug in waffle at the moment)
• Connect vs Fixes
• In Progress items should have someone assigned to them
• Add waffle to the pieces of infrastructure in #14

Stuff that can be hosted in IPFS

• Git repositories
• APT repositories
• NPM
• PIP
• ...

root@sthing2:~# ipfs ping QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64
Looking up peer QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64
Peer lookup error: routing: not found

Got the ID from ipfs bootstrap list

root@sthing2:~# ipfs bootstrap list
/ip4/104.131.131.82/tcp/4001/ipfs/QmaCpDMGvV2BGHeYERUEnRQAwe3N8SzbUtfsmvsqQLuvuJ
/ip4/104.236.151.122/tcp/4001/ipfs/QmSoLju6m7xTh3DuokvT3886QRYqxAzb1kShaanJgW36yx
/ip4/104.236.176.52/tcp/4001/ipfs/QmSoLnSGccFuZQJzRadHn95W2CrSFmZuTdDWP8HXaHca9z
/ip4/104.236.179.241/tcp/4001/ipfs/QmSoLpPVmHKQ4XTPdz8tjDFgdeRFkpV8JgYq8JVJ69RrZm
/ip4/104.236.76.40/tcp/4001/ipfs/QmSoLV4Bbm51jM9C4gDYZQ9Cy3U6aXMJDAbzgu2fzaDs64
/ip4/128.199.219.111/tcp/4001/ipfs/QmSoLSafTMBsPKadTEgaXctDQVcqN88CNLHXMkTNwMKPnu
/ip4/162.243.248.213/tcp/4001/ipfs/QmSoLueR4xBeUbY9WZ9xGUUxunbKWcrNFTDAadQJmocnWm
/ip4/178.62.158.247/tcp/4001/ipfs/QmSoLer265NRgSp2LA3dPaeykiS1J6DifTC88f5uVQKNAd
/ip4/178.62.61.185/tcp/4001/ipfs/QmSoLMeWqB7YGVLJN3pNLQpmmEk35v6wYtsMGLzSr5QBU3

We want to have the gateways at ipfs.io instead of gateway.ipfs.io

This requires

• IPNS proved robust enough for ipfs.io website.
• faster gc on the gateways
• period reboot on the gateways (clear any bad states, etc)

@hosh You have the most knowledge on this.

Some column names for the current sprint:

• Sprint
• Ready

For the backlog column:

• Keep the current name?

dist.ipfs.io and ipfs.io/dist

It would be good to make a list of all available infrastructure in the readme. Currently there are only generalities listed.

What we have:

• http://waffle.io/ipfs/ipfs
• http://ipfs.io/
• http://gateway.ipfs.io/
• ...still compiling list

We'll use this issue to track objects to pin for now. I'll add a file. eventually we'll have a much cleaner way to do it.

(let's edit with a checkbox + point to the commit adding the file)

Recap:

• #9
• ipfs/kubo#283
• ipfs/kubo#1220 (comment)

We should organize our testing infrastructure concerns.

Our constraints are:

• (hard) robust, reliable way to test our code
• (hard) able to run all the tests (with iptb, we no longer need docker)
• (med) smooth, friendly UX for contributors -- including new contributors (travis wins, by far).
• (hard) well integrated with github PRs and commit status indicators (travis wins by far)
• (med) no management overhead on our side (jenkins was a huge pain, that's why we got rid of it)
• (soft) tests take <5min
• (soft) cheap. But doesn't even have to be free. i keep trying to give travis-ci money (they won't take it :( )

Unfortunately, after trying multiple CI systems, always end up coming back to travis. (We used jenkins for a long time and it was a huge pain to manage. life is much better without it).

Close seconds are:

• https://circleci.com/ -- pricey but hear good things. not sure how well they integrate with OSS
• https://codeship.com/ -- they keep improving, but don't take security seriously.

For testing on windows, we could use:

• http://www.appveyor.com/

Currently only ipfs and go-ipfs are enabled.

We should have a base nginx role, and each role that exposes something, should come with its own file for sites-enabled.

• Most promising candidate: http://prometheus.io/docs/visualization/promdash/
• Grafana only has "early support" for Prometheus: http://prometheus.io/docs/visualization/grafana/

CORS regression?

Round-robin over the cjdns addresses:

fc98:424c:b433:d7e2:7ee3:9541:73ff:2cdb
fce3:c53b:c3c5:2f54:8bb0:b6d9:898e:f140
fcfe:eab4:e49c:940f:8b29:35a4:8ea8:b01a
fc4e:5427:3cd0:cc4c:4770:25bb:a682:d06c
fc3d:9a4e:3c96:2fd2:1afa:18fe:8dd2:b602
fcd8:a4e5:3af7:557e:72e5:f9d1:a599:e329
fc29:9fda:3b73:c1d2:9302:31e3:964c:144c
fcdf:a296:afe3:7118:4135:cc0b:ff92:4585

Add https://github.com/ipfs/api

• Waffle.io
• ipfs/ipfs/project-directory

Right now cjdns runs in the host, and all dependent containers use the host network stack.

• containerize cjdns, with --cap-add NET_ADMIN for /dev/net/tun
• give dependent containers the same network stack, with --net container:cjdns

We have a bunch of projects using CI.

1. It would be cool, possibly useful too, to have a single page that gathered them all together. Something like:

   http://ci.pivotallabs.com/

   Which is an instance of this rails app:

   https://github.com/pivotal/projectmonitor

   Works with Circle CI, Travis and others. Could run on a free heroku instance (or wherever).

2. Alternatively, people who care about the overall health of multiple/all builds could pull down status messages to local notifications with CCTray, BuildNotify, etc ( some info on that for Circle here: https://circleci.com/docs/polling-project-status ).

3. In any case, we should probably add passing/failing badges to the projects' READMEs, so you could see build health there.

4. We could leave it as is: build failures should notify the committers, and if you want to check, you can probably peruse IPFS builds in Circle or Travis.

Thoughts? I vote for # 3 + # 1, in that order, with optional # 2 encouragement / instructions.

jbenet commented 7 days ago

We want to put the bots into solarnet. these include:

sprintbot
pinbot - probably should pin to the gateways directly

cc @whyrusleeping @lgierth

in the long run, pinbot maybe should be using an S3 datastore or something.

jbenet commented 3 days ago

pinbot crashed recently. we should bring it into the infra. what needs to be done to run it effectively?

maybe we can run it on jupiter or saturn.

jbenet commented 3 days ago

@whyrusleeping could you outline all the considerations necessary to running these bots?

AFAICT from https://github.com/whyrusleeping/pinbot -- it uses shell, so should "just work" with a standard daemon running in the background, right? there may be some interesting networking magic to do to wire up the containers.

https://github.com/whyrusleeping/pinbot/blob/master/main.go#L102 <-- nice 👍

whyrusleeping commented 3 days ago

Yeah, all you need to do is run the pinbot program on a machine with a daemon running. It would be best to have it on a machine with the most storage available to us. It also is capable of connection stealing updates, so if you modify the code, just run the new binary, it will grab the tcp connection from the old one and take it over to avoid having the bot disconnect and reconnect.

jbenet commented 3 days ago

• add the pinbot binary as a container somewhere
• largest storage? (ideally S3 once 0.4.0 merges)
• updates should gracefully run new bin before destroying the old one

What else?

Remaining questions:

• one machine or multiple?
• what redundancy should we have?
• we could have pinbot0..9, or we could have 1 irc bot that pins to several other nodes

I'm seeing the CORS:

> curl -I https://ipfs.io/ipns/ip
HTTP/1.1 200 OK
Server: nginx/1.9.3
Date: Sun, 06 Sep 2015 03:41:17 GMT
Content-Type: text/plain; charset=utf-8
Connection: keep-alive
Access-Control-Allow-Origin: https://gateway.ipfs.io
Access-Control-Allow-Origin: https://ipfs.io
Access-Control-Allow-Origin: http://gateway.ipfs.io
Access-Control-Allow-Origin: http://ipfs.io
Access-Control-Allow-Origin: http://localhost
Access-Control-Allow-Origin: http://localhost:8080
Access-Control-Allow-Origin: http://127.0.0.1
Access-Control-Allow-Origin: http://127.0.0.1:8080
Suborigin: ipfs.io
X-Ipfs-Path: /ipns/ipfs.io/
Strict-Transport-Security: max-age=15768000

On chrome i get

https://gateway.ipfs.io/ipfs/QmWCobpCNQy9SN19mcauGw4HR5p1KBcTvL6ousavdpJxqt/12/3790/2375.pbf. The 'Access-Control-Allow-Origin' header contains multiple values 'https://gateway.ipfs.io, https://ipfs.io, http://gateway.ipfs.io, http://ipfs.io, http://localhost, http://localhost:8080, http://127.0.0.1, http://127.0.0.1:8080', but only one is allowed. Origin 'https://ipfs.io' is therefore not allowed access.

Looks like we're returning it incorrectly (it should only return for the origin itself-- i.e. if the user's origin is the same one as one of these, then we return it). I think this has to be fixed in go-ipfs, but tagging here as we should make sure to fix it through to deploy.

Track ALL the Issues

Looking at waffle.io. It looks very promising. I played around with the multi-repo linking which isn't perfect but it works. I'm currently waiting upon the creation of ipfs/ipfs to act as a root repo for the IPFS projects.
Tasks for https://waffle.io/ipfs/ipfs:

• Get me permissions to ipfs/ipfs
• Link other repos to ipfs/ipfs
• Correctly set up columns

So they don't get lost, and can be applied to other hosts as well

Requesting https://ipfs.io/ipfs/QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB

Returns

internalWebError: unexpected EOF

Unclear whether this is a go-ipfs issue or nginx etc.

We don't need that many big hosts at the moment.

Can we get one with just the logo and a transparent background?

Discussion conclusion:
We can try https://github.com/ipfs/logo/blob/master/ipfs-logo-1024-ice.png, but @jbenet wasn't sold upon it looking better.

Moved from ipfs/ipfs#62

We'll stick with the scheme of giant spherish space rocks, in order to not introduce another universe of terms (Matt Groening's janitors, in this case)

• Pick names
• Rename droplets
  • Make sure PTR records were updated
• Migrate DNS from $janitor.i.ipfs.io to $spacerock.i.ipfs.io
• Update /etc/hostname on each host
• Reconnect Pinbot so that it picks up the new hostname

Docker currently keeps the complete log of the container, which we throw away anyhow since IPFS keeps its own logs. Docker 1.6 introduced the --log-driver option which we can set to none.

Ansible implements this in 2.0 which isn't released yet. Relevant commit: ansible/ansible-modules-core@718f32a

I've had it with jenkins. it's so painful to work with. I want to move off. It's currently broken-- it isn't building go-ipfs properly.

I really like golang's build system: http://build.golang.org -- it is just https://github.com/golang/build -- ideally we could run our own version of that, equipped with ipfs to avoid re-downloading things multiple times (i.e. so tests get even faster). This sounds non-trivial to setup...

So perhaps in the meantime we could try to use travis for everything we do in jenkins. This may not work, particularly:

• race tests will crawl or may just fail on travis
• not sure if container tests (3nodetest) will work.

The jenkins tests in question are:

• test_sharness_expensive ideally with fuse
• build_docker_image to check it works
• build_all_commits to check all commits build (probably redundant given test_all_commits)
• test_go
• test_3node requires containers
• test_go_race likely to be very slow
• test_all_commits runs make test on all commits.
• test_go_integration runs some integration tests

I think that -- instead of only testing make test on all commits, we should instead:

• for every commit:
  • go test
  • go test race
  • sharness (expensive)
  • build docker image
  • integration tests

This might be as simple as making the repo test all these pieces and using travis.

• Repo moved
• Update ipfs/ipfs/project-directory.md ipfs/ipfs#66
• Add to waffle

We ran into the 10 person google hangouts limit today. i dont think google live hangouts helps much here.

Others have suggested:

• talky.io
• zoom.us
• palava.tv

main requirements are:

• works on all platforms (so ideally webrtc)
• good video / audio quality
• trivial for new users to use (e.g. no need for accounts (link / invite only) )

We shouldn't have them in secrets.yml -- any service's OAuth token is the weakest link, right after our Github accounts.

We want to use GitCop to make sure the commits have the following trailers:

License: MIT
Signed-off-by: user name <email@address>

We might also limit the subject of each commit to 80 characters.

What do you think?

We should have a calendar (or set of calendars) for important dates + community events (talks, etc).

We can setup google calendars @ipfs.io and have it listed to be publicly viewable somewhere. Does that sound good, or are there other services people want to try?

An IRC bot similar to pinbot, but pins only on the storage hosts.

I want to have a full implementation-independent test suite, including things like the current network test, and the sharness tests. This will allow developers of other implementations to benchmark against another test suite.

This requires:

• making a new repo for this -- perhaps impl-tests
• moving sharness, network test, whatever else there.
• the full make test in go-ipfs should still run these tests, so perhaps it can clone the repo locally.

implementations wont have all the same sets of features, e.g. some implementations may be missing gateways, or missing the CLI altogether, or even the API (but still have the protocol implemented). It would be ideal if the test suite allowed implementors to pick the sets of test cases to run. Perhaps something like:

• protocol for full protocol tests
• API for testing conformance to the the api spec
• cli for testing cli tools.

Maybe we should fully decouple the cli in go-ipfs, so that it can be used against other language implementations of the API just as well.

This is a big project. Starting the conversation, as people are itching to get started with implementations in other languages.

It currently happens manually with apt-get...

Add https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin to jenkins to allow admins to run the full test suite on pull requests.

Right now it's publicly available at pluto.i.ipfs.io/prometheus, should hide it behind the cjdns whitelist.

We can't bust 100 percent of existing NATs, but we can provide opt-in relaying via the bootstrap nodes.

Someone needs to go through the New Issues on https://waffle.io/ipfs/ipfs and determine which ones should definitely be closed, moved to the backlog, or moved to the icebox. If the issue is missing any labels then label it. If it's questionable, like backlog vs icebox then get the relevant person involved.

Denylist daemon: https://github.com/ipfs/gateway-dmca-denylist
Notices storage: https://github.com/ipfs/gateway-dmca

@awrelll is setting up a gitter bridge. the goal is to get it to:

• fully bridge gitter ipfs/ipfs with our freenode #ipfs channel.
• make sure ALL messages said on IRC (freenode #ipfs) make it to gitter -- relayed by ipfs-gitter-bot
• make sure ALL messages said on gitter make it to IRC (freenode #ipfs) -- relayed by ipfs-gitter-bot

I regained control of github.com/ipfsbot -- but im thinking that the gitter bridge should be its own account (cause we'll use ipfsbot for other things too), maybe ipfs-gitter-bot

As a note, until those three checkboxes above are robust, i dont feel ok telling people to use gitter. the last thing i want is for people to think we're ignoring them (if their messages arent getting through). an unreliable comm link can be worse than forcing people to use another comm link.

in ipfs/kubo#1261, Gitcop says

There were the following issues with your Pull Request

Commit: ef34768
Invalid signoff. Commit message must end with License: MIT Signed-off-by: .* <.*>

The commit message is

using multistream muxer

* ID service stream
* make the relay service use msmux
* fix nc tests

Note from jbenet: Maybe we should remove the old protocol/muxer
and see what breaks. It shouldn't be used by anything now.

License: MIT
Signed-off-by: Jeromy <[email protected]>
Signed-off-by: Juan Batiz-Benet <[email protected]>

ipfs/kubo@ef34768

lgierth commented 28 days ago

We might wanna separate the IPFS and HTTP traffic and spread it to different hosts depending on the load pattern. One the same hosts, they could still run in the same container. Need graphs to see if it's an issue, or notice when it turns into one.

jbenet commented 28 days ago

Need graphs to see if it's an issue, or notice when it turns into one.

yep, exactly agreed.