Testing Infrastructure about infra HOT 21 CLOSED

Also, relevant exchanges with Travis-CI -- I should say that I'm very impressed with their support:

http://gateway.ipfs.io/ipfs/QmZeCPZPkgSRoLEHKFVGxsEJKiuGye16ekXjtyVdorjdkW/email.html

from infra.

@jbenet could you give more details about the problems with Jenkins?
At my current job we use Jenkins and at my previous job they used hudson (though they started switching to buildbot), and my experience is that it works fairly well and is not so painful to manage.
But maybe it depends on the setup, how the build slaves are configured or managed?

from infra.

Also when you say "tests take < 5min", this is for one commit.
But it could be nice if we could test a 10 commit long branch without having to use 10 branches.

And about running all the tests, does that means FUSE tests too?

from infra.

@chriscool i think the issue with jenkins (at least in my experience) was that it was confusing and we (well at least, i) didnt quite understand how to get it all working. It wasnt as simple as we wanted

from infra.

@whyrusleeping maybe I can have a look at it on Saturday or Sunday.

from infra.

@chriscool that would be really great :) I'd love to have a more full test suite running again

from infra.

i found jenkins unintuitive, complicated, and unpleasant to use. if someone else wants to take care of it and have it running-- that's fine, but i got tired of doing it. I don't want to have to look at it. travis works much better so far. travis is hyper clear-- everything is very simple, intuitive, or very well documented. it's also pleasant to use.

I would really, really love to use this: http://build.golang.org -- https://github.com/golang/build -- i just took a look and it doesn't seem very complicated.

On the branches, ideally we could convince travis to do it automatically for us, but https://github.com/jbenet/git-push-each is not bad. We also get the indicator per-commit, which normally wouldn't be there. the last thing needed is a bot that automates it. it's not a big deal to have extra branches that are added and removed. After all, branches are very, very cheap.

from infra.

At the bottom of https://github.com/golang/build there is:

If you wish to run a Go builder, please email [email protected] first.
There is documentation at https://golang.org/wiki/DashboardBuilders but
depending on the type of builder, we may want to run it ourselves, after you
prepare an environment description (resulting in a VM image) of it. See the env
directory.

I wonder if we should contact them?

from infra.

Yeah perhaps! I think they mean running a gobuilder to help with building go itself. They probably don't want to take on the challenge of supporting other groups, but they may at least help with pointers on how to set it up to test other things :)

—
Sent from Mailbox

On Sat, May 16, 2015 at 1:51 PM, Christian Couder
[email protected] wrote:

At the bottom of https://github.com/golang/build there is:

If you wish to run a Go builder, please email [email protected] first.
There is documentation at https://golang.org/wiki/DashboardBuilders but
depending on the type of builder, we may want to run it ourselves, after you
prepare an environment description (resulting in a VM image) of it. See the env
directory.

I wonder if we should contact them?

Reply to this email directly or view it on GitHub:
#20 (comment)

from infra.

Also can we run https://github.com/jbenet/git-push-each in another Travis job? Is there a limit in the number of Travis jobs? And what is the difference between the 2 jobs we have (pr and push)? Is it just that one is run on the PR branch and the other one on the master after the branch is merged?

from infra.

And about gobuilder, yeah it looks like they mean running a gobuilder to help with building go itself.
This is a bit worrying because perhaps it means that gobuilder is not well suited to build something else. We will see when we contact them.

from infra.

@chriscool

take a look at:

• http://docs.travis-ci.com/user/build-configuration/
• http://docs.travis-ci.com/user/speeding-up-the-build/
• http://docs.travis-ci.com/user/build-configuration/#The-Build-Matrix

seems like we could break up the "go test" and "sharness" parts with different build targets. maybe:

env:
  - TEST_SUITE=go_test
  - TEST_SUITE=sharness

script: "make $TEST_SUITE"

or something.

is there a limit in the number of Travis jobs?
probably, not clear how big. we can ask them if we hit a limit.

what is the difference between the 2 jobs we have (pr and push)? Is it just that one is run on the PR branch and the other one on the master after the branch is merged?

Correct-- the idea is to see whether the errors are on the branch or in merging with master. i think they started doing that automatically because people pointed out that was a problem.

from infra.

Circle CI is awesome. It's free for open source. I use it exclusively for new ruby and node projects, including continuous deployment to Heroku and github pages. I haven't used it with docker or go. I used Travis before Circle, and I'd never go back. I can't point to features Circle has that Travis doesn't; but everything feels even cleaner, more lucid, more usable. They are always adding new features, and are eager to hear what users want.

from infra.

Great to see that go-ipfs is passing now on Cirlce CI (nice work @jbenet).

I know that we've installed both Circle and Travis on a few projects. Which do people prefer? It would be great to standardize, especially to move forward on aggregating CI for app IPFS projects into a single place.

from infra.

@chriscool did you ever email to golang/build folks?

from infra.

No I didn't I can do it these days.

from infra.

Ok, I sent an email entitled "Running a Go builder for IPFS", but it looks like it didn't make it to the golang-dev group:

https://groups.google.com/forum/#!forum/golang-dev

from infra.

the groups are moderated -- takes some time for mods to let the messages through

from infra.

In this thread i started: https://groups.google.com/forum/#!topic/golang-nuts/kRq8Ow4qEeU bradfitz says this is pretty doable. I think we should do it. cc @whyrusleeping

from infra.

I don't know much about it but I am ok with working on this.

from infra.

Closing in favor of #100

from infra.