Add more options to be up to par with other CI solutions about docker-buildkite-plugin HOT 8 CLOSED

Thanks for the issue, and you're right, it's pretty basic at the moment. Have you seen the docker-compose plugin?

from docker-buildkite-plugin.

Yeah, I've seen it, but after using various CI, I find docker's buildkite integration is much more complicated than it should be. I don't think requiring a docker-compose file for a simple build is great.

When we tried concourse, it took a few minutes to set it up to build simple docker images. Same with GitlabCI.

With buildkite, we had to fork and build our own plugin =\ It actually took us a few day to make it work. I can imagine others coming from other CI and feeling the same, and we would gladly like to help you with that, cause we think Buildkite is super dope and powerful.

Our use case: we wanted to run the agent inside of kubernetes, so we can take advantage of autoscale. With the existing plugin, it's just not possible.

Here's our version: https://github.com/playerme/smart-ci-buildkite-plugin/blob/master/hooks/command

There's a lot of useless stuff (cause it's just for us), but I figured the option to mount docker socket and extra volumes would be very useful.

from docker-buildkite-plugin.

Thanks for the background + context! I understand. I want to hear more about how you hooked up the autoscale too.

In general, I think supporting a bunch more of the docker-run arguments makes total sense. We'd gladly accept PRs for them, ala #6

It's probably worth breaking them up into separate PRs for each option that's added if possible, only because they're easier to merge and discuss bit-by-bit.

from docker-buildkite-plugin.

It's probably worth breaking them up into separate PRs for each option that's added if possible, only because they're easier to merge and discuss bit-by-bit.

Yes, I'll break them up in small pieces and add the correct documentation at the same time.

I want to hear more about how you hooked up the autoscale too.

I guess I could contribute to documentation about that, but we've been running into some problems as of late, and I'm not sure if it's because of Buildkite, our plugin, or our git repository. We made the autoscale works but for anything more than 12 concurrent agents, and our "random error rates" goes thru the roof.

Example types of errors:

• Errors out on git clone
• Randomly freezes because it's an unverified SSH host and requires someone to input "yes" (even though known_hosts is passed).
• Randomly errors out and fails.

We're still investigating to find the source of the problems. My gut feeling tells me it's because we're trying to cram too many agents in 1 node, but it still happens whether I have 12 nodes or 1 =.

For a quick rundown of how we autoscale:

• Each agent has a CPU and Memory request
• We add the Horizontal Pod Autoscaler (https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)

For now, it's totally based on CPU/RAM but it's not really accurate - some jobs require less CPU and RAM than others jobs. In the future, we plan to add custom metrics and autoscale based on running pipelines.

Anyways, I'll start the PR and tests this week. :-)

Cheers

from docker-buildkite-plugin.

Awesome, much appreciated.

Thanks for all those details! I was wondering if it was custom metrics or CPU… I've heard others using the horizontal pod autoscaler with CPU — I discussed it with @rimusz a while back. Would be awesome to have it use something based off metrics generated by https://github.com/buildkite/buildkite-metrics

from docker-buildkite-plugin.

Oh wow, had no idea that existed. That would also help for our prometheus exporter we want to build too.

from docker-buildkite-plugin.

Nice! We should add a prometheus backend to it.

from docker-buildkite-plugin.

@toolmantim

Added #8, #9, #10. None of the patches are breaking, every existing config should work.

I omitted other patches because #6 covers them, though mine were named differently (instead of naming it shell, I named it entrypoint which is the correct name in the docker world)

In my version of #6, the default entrypoint is bash -c to keep backward compatibility. If you prefer that, let me know, I'll make a PR for it.

Once these are merged, I can start the patch for hooks. Basically, buildkite hooks don't really run inside of docker, which makes it problematic when you need to fetch secrets and stuff. pre-command and post-command both runs outside of docker, instead of inside.

from docker-buildkite-plugin.