Revisit immutable tag propogation for workflow(-dev/-canary) chart about charts HOT 6 CLOSED

Why are we using canary builds at all? Not only are mutable tags a widely recognized Docker anti-pattern, they're a disaster from a bug reporting standpoint.

Unfortunately, in practice, the e2e tests aren't 100% reliable and so some perfectly good component changes (immutable tags) fail to get into the master version of the workflow-dev chart in a timely manner.

Why don't we put the effort into fixing this?

from charts.

my $0.2 - we should put effort into speeding and determinizing (yes, I made that word up) our e2e tests, and remove canary. Right now, it just seems like these tags are a band-aid on this larger problem of e2e testing...

Real example: we don't go build our CLI binaries and upload them to deis-canary on Bintray before we run go test.

from charts.

I vote option 3 but keeping the workflow-dev name - Stabalizing the development workflow at this point is important

For controller (as an example) locally I only use the immutable tag to update my own registry when I'm developing so that I know what branch I was working against anyway.

from charts.

My 2 cents:

I like the current model, because it is a persistent reminder that our E2E tests are currently non-deterministic. Recent work on our E2E implementation has improved runtimes, so I think it's reasonable to expect the merger to investigate failed post-merge E2E test failures (retry, take notes, etc), and to use that investigation towards a more deterministic E2E paradigm.

I don't think 100% E2E tests are realistic, but I dislike the idea of building rather complicated processes to account for mysterious E2E failure because however clever we are in working around such mysteries, we will end up in the situation where mystery is institutionalized: "E2E tests are flaky and we don't trust them and that is just that." Now is the time to at least aim for better than that.

from charts.

I'm happy with option 1 and fixing up CI to deploy git SHAs in a timely manner through enhancements to the test suite. If we can make the tests 95% reliable then I'd be completely happy with a few hiccups here and there. That's just the nature of end-to-end test suites, unfortunately. Network latency, services going down, etc. All can eventually be fixed.

from charts.

Thank you for the feedback. It sounds like the consensus is to keep our current process of promoting immutable tags in tact, maintaining focus on e2e determinism.

Many have questioned the process of pushing mutable/canary tags of components along with immutable, therefore deis/workflow-e2e#152 was created to track the proposal to remove this process and let that particular conversation hash out there.

from charts.